Thoughts on mentoring within the analytics profession

Our careers and lives follow unique trajectories and structures. While we all have our own paths, it is helpful to have people who have gone ahead on similar paths to share experiences and thoughts on the future. Part of our professional development are mentorship relationships, which can be done in a wide range of settings, relationships, and time frames.

I am going to define mentoring as a longer term, unstructured professional relationship, with the focus of the relationship being the personal growth of the mentee.  Typically, the basis of the relationship is that the mentor has gone on a path that the mentee is on themself, and the insights of time may be helpful for the mentee's development.

One thing that distinguishes mentorship relationships from other professional relationships is that mentorship relationships are holistic.  They look more than just the task at hand, or even a job position. The mentorship relationship may be career focused, but it will look at the whole person, and will recognize that overarching goals can change with life events, even life events outside their occupation. So, while a supervisor/manager can be a mentor, this is really not apparent until after the manager relationship has ended, and the relationship has become larger than the roles both individuals had when the relationship started.

As we all have unique life paths, we cannot expect that any one person has gone on the same path that we are on, but mentors bring not only their own life experience, but also the experiences of those whom they have lived life alongside. They have seen the decisions and choices of others, and how those decisions have advanced the goals, or not. They have seen people whose lives have taken them on different paths, and so have a broader view on what the future can hold than those whose view of the world is from the relatively structured life of home and school.

What topics come up? The focus on a mentorship relationship is on the growth of the mentee. In the context of technical professionals, this is the professional growth, but as part of a full life. So, with an understanding of the long term goals of the mentee, it can be working through broader issues on a project, such as other points of view.  It can be soft skills or relational skills working with co-workers, superiors, juniors, or outside colleagues (customers, business partners, etc.).  It can be suggestions on how to stretch as a person, to see and work through things from a broader perspective, and the skills needed to do this.  A mentor can be a sounding board, providing different points of view (especially on the behalf of people who may not be good at communicate their point of view). It can be how to handle work/life balance, looking at a whole person. It could also include looking at alternative paths, that different positions or even career paths may be more suited for the goals of the mentee. 

How does a mentorship relationship start? Like all relationships, you can never tell if a relationship is going to be long term at the beginning.  But you have to begin somewhere. A first conversation is often about a particular topic, one that is of mutual interest. (and this initial meeting is sometimes arranged by organizations such as a company or a professional organization trying to promote mentorship among employees or members). After the first few conversations about that first topic, you should have observed if the relationship is broader than that first topic, and you can talk about if you want to continue meeting about topics as they come up.

What does the mentor get out of this relationship? Typically, people who are in mentoring relationships also have other rich relationships, which is how they get the background that makes them valuable as a mentor.  Over time, the relationship becomes driven by both concern and curiosity about the other's experiences in life. Often that includes issues that are more apparent to someone at an earlier stage of life or career. A mentorship relationship can then become one of an ongoing set of relationships that makes up a life well lived, and the ultimate hope, even when it is not an expectation, is that a relationship be one that lasts.

Do mentorship relationships last?  Sometimes. Organizations such as workplaces and professional societies will often organize mentorship relationships, but these are always based on a topic of interest in the moment, and these relationships typically start out with short term boundaries. But, like all relationships, a short term relationship is what has potential to broaden into something longer. Does the relationship broaden beyond the topic where it was started?  Do conversations evolve organically and feel natural when they branch into new topics? Over time, can the relationship feel like something that lasts as both sides grow and change (as all growing people do). So the transition from a formal, temporary relationship with a defined schedule and defined boundaries changes into something more long term and fluid. And a mentor/mentee relationships begins to feel more like professional colleagues, each moving through life and careers on adjacent paths. 

Can mentorships relationships be informal? Yes, in the sense friendships are informal. In professional society meetings, it is common to see someone and immediately follow up from a conversation from a year ago, just like old friends. So you can have a relationship where you only see each other on occasion, but immediately pick up where you left off, just as old friends do. But the key is the long term relationship, that the conversations are about growing people, not only about topic at hand.

Are there aspects of Analytics that mentorship relationships are especially helpful? One area are the soft skills, the skills of working with colleagues, managers, and customers that is not part of the standard training of a technical professional. A mentor can relate to what the other person may be thinking and help the mentee develop that sense of empathy for others that make them more effective professionally.  A second aspect is dealing with the hype that often accompanies the profession. The most recent example is the rise of Generative AI, but similar waves of publicity occurred around deep learning, big data, and machine learning in general. A mentor can place new ideas and concepts in the context of everything else a mentee knows, in contrast to teachers or thought leaders whose responsibility at any given point in time is single topic focused. A third aspect is a sense of what a mentee may need to be a well rounded professional. Training programs and classes tend to be singularly focused with a specific goal, but professional growth needs to be holistic, and designing such a path needs the attention of a person who is looking at the whole person.

Mentorship presents the potential of a valuable relationship, fostering personal and professional growth through a holistic and potentially long-term connection. It goes beyond task-oriented guidance, embracing the mentee's whole person, from developing crucial soft skills and navigating career paths to contextualizing industry trends. While it can begin focused on specific topics or within formal programs, at their best mentorships evolve into enduring relationships, offering mutual benefits and enriching the lives of both mentor and mentee. In dynamic fields like Analytics, such relationships are particularly vital, providing the comprehensive support needed to cultivate well-rounded, effective professionals in changing times.

If you are interested in mentoring relationships, I would look to your professional society. If you are in analytics, I would recommend you look at INFORMS and their mentoring programs (Video on the value of mentoring in analytics)  It is a professional society for advanced analytics (broadly defined) and is vendor, tool, and methodology neutral, which is important for a field that sees major changes over the course of decades.

Subject domains that lead to failure in large language models output

At the 2025 YinzOR conference I was talking with Léonard Boussioux about types of domains where large language models (LLM) have a tendency to fail, and other conversations encouraged me to write this down.

There are stories of the early days of aviation, where a test pilot would come back and learn that his plane had cracks, and were delighted because that meant that they were learning the limits of the aircraft.  In that spirit we want to look for domains where the foundations models will give poor results, so that those developing applications can look for potential failures and design applications and train users to be attentive for errors.  For this discussion, the cause of the errors are the data used to train the foundation models.  Like other deep learning based models, to uncover categories of errors, we look at the training data.

Large language models tend to fail due to inability to work with nuance and naivete.  My friend Polly Mitchell-Gunthrie describes LLMs as unable to work with context, collaboration, and conscience.  I describe problems in LLMs as failures in nuance, naivete, and novice problems.  Again, this is due to how the foundation models are trained (effectively all publicly available text), so these are social problems, and my not be solvable in this real of LLM based AI.

Novice problems are due to the characteristics of what is available on the internet.  The majority of information on the internet is aimed at beginners. (computing topics are significant exceptions to this) So there is a lot of information that rises to the equivalent of an introductory sequence in college.  So it has a body of knowledge.  But using a body of knowledge that is targeted at introductory level leads to nuance, naivete, and novice errors.

Nuance issues are probably the most recognized.  Nuance comes into play in subjects were details matter where the answer in a specific situation is not the same as the standard case.  When given a setting, an LLM (like a novice) will take the information provided in the prompt and find other sources that include the same information and come up with an output (answer).  However, and expert would take information and fit it into an applicable framework.  Then, the expert will recognize that there is missing information that influences the final answer and ask for that information. Similarly, when considering other references, the same framework tells the expert the extent of applicability of that reference.  An LLM only matches text in the prompt with the references, so will not always check that the context of the reference matches the context of the setting of the user.  These types of issues lead experts to reach very different conclusions than people who are new to a domain, and the LLM  tend to act like novices here.  As an exercise to help people identify domains where LLMs do badly, I ask people to pick a topic that they know well, but not through textbooks or classwork, and not computer related (this tends to lead to topics that they know experiencially or through true research).  Most people identify a hobby, my manager did this exercise with his master thesis topic.  Another variation of nuance are details that frequently occur together, but are not the same.  Since the LLM works by probablistically choosing words that occur together, it can often try to combine related topics or words that should not be.  A frequent example of this is in anatomy, where LLMs trained on medical texts will often conflate the names of two body parts and into a body part that does not actually exist.

Naivete occurs when someone is in possession of facts, but does not recognize the consequences of those facts.  For an LLM, it is easy to take a prompt, then from references that match that prompt, identify other facts/details that are typically associated with the information provided by the prompt.  But unless it finds references that explicitly spell out the consequences of a particular collection of facts, the LLM will not provide the consequence.  As an example, my then 10 year old daughter had written a story that was set in a domestic setting in the United States during the 1860s (U.S. Civil War era). So when I ran her through the exercise of a topic that was not well known, she asked the Generative AI about an aspect of domestic life, specifically methods for starting fires.  Her comment was that the generative AI gave details that as far as she could tell were all true. But, it did not provide an important consequence.  When given the same set of details, a modern day chemist would mentally translate the 19th century terms to modern day counterparts, and immediately recognize that it contains all the ingredients to cause an explosion. And in real life this is what happened so there are very few examples of this technology in museums, because they all exploded. And my daughter regarded that knowing a technology meant for use in domestic (home) life had a tendency to explode to be an important detail and the LLM not reaching that conclusion to be a failure.

Another type of novice error are exceptions and crossing domains.   Many domains will teach general frameworks and rules of thumb at the introductory level.  They are intended to help practitioners succeed and to avoid common pitfalls.  However, past the introductory level practitioners learn the reasons behand the framework and rules, either from deeper training or through experience, so experts will know the exception to the rules or when to modify rules based on the particular circumstance at hand.  This is even more important in cases where multiple domains are involved, which is common outside controlled environment such as academic or teaching environments.  In this case, the standard rules for the multiple domains can conflict.  Experts will resolve this both by establishing exceptions based on the circumstance, but also looking at the ultimate goal or intent of the activity, and break or bend rules based on which rules interfere with the goals or the mission.  But they don't completely through out the rules, experts will keep in mind the intent of the rule and ensure that the intent is addressed.  When LLMs are given both the rules of the domains as well as history of prior activity, the LLMs will often identify the fact that rules are broken, and no longer follow the rules, which leads to poor outputs that do not respect the issues that arise with these domains in practice.

LLMs are especially handicapped when there are intersecting domains.  When articles or other texts are written or published, the general rule is to have anything you write/publish be on a single topic, which makes it easier to identify the target audience and for the target audience to find your work. Topics that are within intersecting domains tend to be niche topics, and are both difficult to get published and difficult to find. An thus less likely to be included in the foundation models training data.  Another area that is not found in published texts are failures.  In many domains, expertise is developed through experiencing failures. However, these domains tend not to document or publish the failures that experts learn from because of potential of repercussions or public disapproval. And if these are not published, they will not be available for training foundation models.

The purpose of this exercise is to make Generative AI useful. And to be useful the ones who work with Generative AI models have to be able to recognize and look for so that they can screen Generative AI output for other types of errors.  For example, my now 11 year old daughter continues to identify errors in Generative AI output ranging from trivial to profound, and because she has this ability, I have no concerns about her use of Generative AI.  Same with my colleagues, once they have experienced identifying errors in AI (and this holds for machine learning models as well), they are able to identify future errors and react appropriately, and not taking the outputs of AI as automatically true.  And this leads to more productive use of AI.

Why take opportunities for public speaking as an analytics professional

For many of us in technical fields, public speaking often feels like a skill we left behind in school or perhaps dusted off for job interviews, especially if our roles involved training or teaching. Once we're in the professional world, the focus tends to shift solely to our day-to-day tasks, and public speaking opportunities seem to dwindle. However, effective communication is crucial for professional growth, and unfortunately, workplaces don't always provide sufficient feedback on technical presentations.

This is where engaging with local professional communities can be incredibly valuable. While I've had the privilege of speaking at professional society conferences, I've also found immense benefit in giving talks within local technical organizations. Many metropolitan areas are familiar with these as "Meetups," named after the platform that serves as their online home. These local speaking engagements offer distinct advantages compared to academic talks or large industry conferences.

Low-Stakes Practice Environment

---

One significant benefit of giving technical talks locally is the opportunity for low-stakes public speaking practice. These communities are typically smaller, comprising individuals genuinely interested in professional development. Because many members also use these meetings as a platform to share their own insights, the environment is inherently supportive and sympathetic. It's a space free from the competitiveness that can sometimes arise when individuals are trying to build a reputation or feel they're in direct competition. This fosters a very friendly atmosphere for honing your presentation skills, where attendees genuinely want to see you succeed.

Sharpening Your Communication

---

Secondly, preparing a talk for a public audience compels you to think critically about what truly matters. In a work setting, it's easy to gloss over foundational concepts because everyone involved in a project is assumed to have that background. In a public forum, you're required to identify the essential information and ensure you cover it as necessary background. This is particularly true for work-related topics when you might need to use public datasets (as most companies don't permit the use of proprietary data for more informal talks). This process forces you to consider what's important for your audience and what's technically crucial. It's an excellent exercise in organizing your thoughts and effectively communicating them, a skill that translates seamlessly back to your work when you realize not everyone on your team has the same background knowledge.

Building Professional Community

---

Finally, these local groups are instrumental in fostering community. Recent articles in local Pittsburgh publications have highlighted the increasing difficulty of forming social connections after school, and professional colleagues, while valuable, often don't entirely fill this gap due to shorter average tenures at companies and the inherent limitations of work-only relationships. Professional organizations offer the unique advantage of being specific enough to align with shared interests, yet broad enough to expose you to ideas beyond your immediate work. Giving a talk provides a natural reason for others to engage with you, sparking discussions and building relationships that can extend far beyond any single job.