What is needed to work in the age of Generative AI

Last week I was at CMU-Heinz for a fireside chat type event with students in the various MS in Analytics programs there. One question that I got was what were the skills needed to succeed in an environment with AI, and even into the future.  Then I spoke about being able to program because you need to learn how to think deliberately, being able to connect technical capabilities with end business needs (because this has always been how analytics fails), and as I think about it more having a better understanding of knowledge. Because if you believe that your education and training is about learning sets of facts and recipes, AI will eat you alive. So your understanding of your field has to be greater than facts and procedures.

First, why learn computer programming when AI can write code faster than you. Microsoft has a set of studies showing a high (40%) rate of errors, yet their programmers also say they are more productive. Because, as I use AI at work, I find that it is helpful in creating good structure and framework scaffolding, especially when I have to context switch (I regularly switch between three data stacks at work, each of them have many people who spend all their time in one) or if I am applying methodologies new to me or my organization.  But because I am competent, I can correct it as I go, and the fact that there were originally errors is not a big concern, because I was going to revise everything anyway

Another reason to learn programming is you learn to think in a different way. The ancient Greek philosophers had students learn geometry before philosophy. Not because geometry and math is beautiful (even though they are), but because with geometry comes proofs. And geometric proofs is about how much you can understand starting with a minimum amount of assumptions (Euclid's five axioms). And you now have experience in determining an objective truth, no appeals to authority, no claims of different point of view. And your logic is in the open, to be critiqued on their own merits. Far different from my friend in grad school who claimed that perception is reality. And only then were you fit to move into the realm of ideas, where even facts have to be evaluated.

Programming languages differ from natural languages in their precision. Every statement has a single clear meaning. And this is different than natural languages, where the ambiguity of human life plays a role. So to work with anything regarding computers, it is helpful to recognize that computers will work with language in different ways we do, it handles ambiguity differently than people, and how it will use randomness to handle the difference (which is key to how Generative AI works).

The next is the link between the capabilities of technology and the needs of the business.  According to everyone who has studied project failures in depth, failed communications between the business partner and the analysts is the biggest cause of project failure. And data projects have a failure rate between 80-90% (this range has been persistent in studies over decades in the data world, and it is consistent across definitions of failure and different segments in data analytics, data engineering, or data reporting (dashboards)). Being able to understand the business needs of end customers as well as understanding potential classes of technology solutions leads to asking better questions and getting value out of the applications of technology.  The main reason for breakdowns in communication is ego and arrogance.  From the technology side, there is often a belief that the customers are idiots who do not know what they want, so the technology people should just build something and pitch it back to the customers.  This is mirrored by business people who think that technology is a a turnkey product so they should not interact with the people who are creating the solution.  A third variation is when upper leadership decides to act as an intermediary between the analysts and the end customer. The logic here is generally that the leader believes both the technologists/analysts and the end customer have no communications skills, therefore the leader will handle all of the communications and give the requirements to the analysts.  All of these are wrong. Especially in anything involving data, details matter, and the entire project involves discovery of details that no-one realized were important at the beginning. So the analyst and the end customer need to be regularly reviewing these discoveries, and adapting along the way.  And only the end customer (because they are closest to the problem on the ground and know what kinds of actions can be taken) and the analyst (because they will be representing the detail in models and they know what the range of alternative models can do) together can make those decisions.  Without that direct communication (potentially facilitated by someone who knows both sides), a project falls into the trap of solving the wrong problem. And this requires people who understand purpose and can determine the impact of nuance, both of which Generative AI does badly in.

The third category of future work is understanding your field.  Computers are very good at retrieving facts, if those facts are in its knowledge base. Gen AI is better than prior technologies because it is not as sensitive to getting the wording precise.  Computers are also very good at following instructions if those instructions are given. (people also tend to be better at things when they are given good instructions). So, if this is the extent of your subject expertise, you are in trouble.  In software development, there is actually a very large workforce like this, whose careers are built on the ability to fill out a given framework or instructions. But if your place in the world is built on more than knowing facts or following recipes, if there is actual understanding that has to be applied on a situation specific basis, there is still room for you. Without that understanding, an organization can execute perfectly, a solution to the wrong problem. Which is worthless. So you need the level of understanding that allows for good judgement, and you need to be working in an organization that allows for its employees to use that judgement.

A last criteria is based on a number of conversations I've had.  Many people express that they believe in the answers Generative AI gives because of the massive investment these companies have made, the smart people they have hired, and the belief that these companies would ensure correctness. I had to explain that these are industries and communities that have historically claimed they had no interest in accuracy or correctness. And until recently, viewed the paying customer as king and only sought to full the demand. Ethics was not part of the conversation.  And what they delivered, did not come with guarantees other than it does what it does.  This attitude that you did not question the authority that came with wealth and success is the first thing that has to be broken before people can use Gen AI productively.  Both of my kids do it.  I design the rollout and presentation of projects at work to make sure my business partners who are using Gen AI view its output skeptically and looking for specific types of flaws.  As an avid reader of science fiction over the years, much of which addresses AI as part of society, and worry much less about the power of AI than I do about people who use the output of AI without being critical thinking. It is the kind of following that leads people to enact policies without analysis, and punishes people. And the outcomes are the fault of the people who followed the AI. Because AI has no goals, purpose, or conscience beyond that of its user.

Thoughts on mentoring within the analytics profession

Our careers and lives follow unique trajectories and structures. While we all have our own paths, it is helpful to have people who have gone ahead on similar paths to share experiences and thoughts on the future. Part of our professional development are mentorship relationships, which can be done in a wide range of settings, relationships, and time frames.

I am going to define mentoring as a longer term, unstructured professional relationship, with the focus of the relationship being the personal growth of the mentee.  Typically, the basis of the relationship is that the mentor has gone on a path that the mentee is on themself, and the insights of time may be helpful for the mentee's development.

One thing that distinguishes mentorship relationships from other professional relationships is that mentorship relationships are holistic.  They look more than just the task at hand, or even a job position. The mentorship relationship may be career focused, but it will look at the whole person, and will recognize that overarching goals can change with life events, even life events outside their occupation. So, while a supervisor/manager can be a mentor, this is really not apparent until after the manager relationship has ended, and the relationship has become larger than the roles both individuals had when the relationship started.

As we all have unique life paths, we cannot expect that any one person has gone on the same path that we are on, but mentors bring not only their own life experience, but also the experiences of those whom they have lived life alongside. They have seen the decisions and choices of others, and how those decisions have advanced the goals, or not. They have seen people whose lives have taken them on different paths, and so have a broader view on what the future can hold than those whose view of the world is from the relatively structured life of home and school.

What topics come up? The focus on a mentorship relationship is on the growth of the mentee. In the context of technical professionals, this is the professional growth, but as part of a full life. So, with an understanding of the long term goals of the mentee, it can be working through broader issues on a project, such as other points of view.  It can be soft skills or relational skills working with co-workers, superiors, juniors, or outside colleagues (customers, business partners, etc.).  It can be suggestions on how to stretch as a person, to see and work through things from a broader perspective, and the skills needed to do this.  A mentor can be a sounding board, providing different points of view (especially on the behalf of people who may not be good at communicate their point of view). It can be how to handle work/life balance, looking at a whole person. It could also include looking at alternative paths, that different positions or even career paths may be more suited for the goals of the mentee. 

How does a mentorship relationship start? Like all relationships, you can never tell if a relationship is going to be long term at the beginning.  But you have to begin somewhere. A first conversation is often about a particular topic, one that is of mutual interest. (and this initial meeting is sometimes arranged by organizations such as a company or a professional organization trying to promote mentorship among employees or members). After the first few conversations about that first topic, you should have observed if the relationship is broader than that first topic, and you can talk about if you want to continue meeting about topics as they come up.

What does the mentor get out of this relationship? Typically, people who are in mentoring relationships also have other rich relationships, which is how they get the background that makes them valuable as a mentor.  Over time, the relationship becomes driven by both concern and curiosity about the other's experiences in life. Often that includes issues that are more apparent to someone at an earlier stage of life or career. A mentorship relationship can then become one of an ongoing set of relationships that makes up a life well lived, and the ultimate hope, even when it is not an expectation, is that a relationship be one that lasts.

Do mentorship relationships last?  Sometimes. Organizations such as workplaces and professional societies will often organize mentorship relationships, but these are always based on a topic of interest in the moment, and these relationships typically start out with short term boundaries. But, like all relationships, a short term relationship is what has potential to broaden into something longer. Does the relationship broaden beyond the topic where it was started?  Do conversations evolve organically and feel natural when they branch into new topics? Over time, can the relationship feel like something that lasts as both sides grow and change (as all growing people do). So the transition from a formal, temporary relationship with a defined schedule and defined boundaries changes into something more long term and fluid. And a mentor/mentee relationships begins to feel more like professional colleagues, each moving through life and careers on adjacent paths. 

Can mentorships relationships be informal? Yes, in the sense friendships are informal. In professional society meetings, it is common to see someone and immediately follow up from a conversation from a year ago, just like old friends. So you can have a relationship where you only see each other on occasion, but immediately pick up where you left off, just as old friends do. But the key is the long term relationship, that the conversations are about growing people, not only about topic at hand.

Are there aspects of Analytics that mentorship relationships are especially helpful? One area are the soft skills, the skills of working with colleagues, managers, and customers that is not part of the standard training of a technical professional. A mentor can relate to what the other person may be thinking and help the mentee develop that sense of empathy for others that make them more effective professionally.  A second aspect is dealing with the hype that often accompanies the profession. The most recent example is the rise of Generative AI, but similar waves of publicity occurred around deep learning, big data, and machine learning in general. A mentor can place new ideas and concepts in the context of everything else a mentee knows, in contrast to teachers or thought leaders whose responsibility at any given point in time is single topic focused. A third aspect is a sense of what a mentee may need to be a well rounded professional. Training programs and classes tend to be singularly focused with a specific goal, but professional growth needs to be holistic, and designing such a path needs the attention of a person who is looking at the whole person.

Mentorship presents the potential of a valuable relationship, fostering personal and professional growth through a holistic and potentially long-term connection. It goes beyond task-oriented guidance, embracing the mentee's whole person, from developing crucial soft skills and navigating career paths to contextualizing industry trends. While it can begin focused on specific topics or within formal programs, at their best mentorships evolve into enduring relationships, offering mutual benefits and enriching the lives of both mentor and mentee. In dynamic fields like Analytics, such relationships are particularly vital, providing the comprehensive support needed to cultivate well-rounded, effective professionals in changing times.

If you are interested in mentoring relationships, I would look to your professional society. If you are in analytics, I would recommend you look at INFORMS and their mentoring programs (Video on the value of mentoring in analytics)  It is a professional society for advanced analytics (broadly defined) and is vendor, tool, and methodology neutral, which is important for a field that sees major changes over the course of decades.

Adventures in core.logic: learning clojure and logic programming with help from Gen AI

 

This past month my project has been to learn logic programming, and as a vehicle to do this, learn clojure (again).  For those who are not computer scientists, logic programming is one of the four main computer programming paradigms:  procedural (what most people learn in an introductory programming class), object oriented (what most computer science programs and professional programmers aim for, Java, C++, C#, Ruby are all examples of OO languages), functional programming (Lisp and its relatives), and logic programming.  The closest most people get to logic programming is SQL, which is declarative and works by expressing the outcome, but not the steps to get there.  The most well known language is Prolog.  A more recent expression of logic programming, is miniKanren, which is a Domain Specific Language originally implemented in Scheme, but there are other implementations, whose quality seems to be related to how well functional programming is implemented in those languages.  This essay looks at (1) learning clojure (a Lisp that runs on the java virtual machine, (2) learning logic programming (3) learning core.logic, which is the implementation of miniKanren on clojure, and (4) using Generative AI to help with all these things.

This is my second exposure to Clojure, which is a Lisp (a functional programming language) that runs on the Java Virtual Machine. The big draw is that it provides a functional programming way of working that allows use of all Java libraries.  As a data scientist, the advantage of functional programming is that this is a much better style of programming when doing data manipulation. For example, using R with the tidyverse is functional style programming in that you perform operations on data frames that return data frames, and this allows the use of piping/sequencing of functions that conform to this pattern. (Pandas in Python is a flawed version of this as not all functions in Pandas follows this rule)

My first run with Clojure was around 2014 (so says my Github timeline). At the time the Incanter project was trying to establish it as a data analysis environment on the JVM. With the goal of being used in corporate IT departments that had standardized on the JVM (which places obsticals to using Python or R).  And it was good enough that I had written a model and associated analysis in Clojure for an attempted startup (a clean implementation which was not done at any of our home organizations). But the Incanter project stalled. And more recently a broader effort to provide data analysis/scientific computing capabilities into Clojure shows promise. Scicloj.  One standard mantra that I can confirm.  Lisp makes the claim that it has very little syntax, it is easy to learn.  And I would agree. After almost 10 years, a short online course and a review of some books I had from 10 years ago I was pretty up to speed.  Because when everything is a list, the question then becomes what is the form of that list for the task/function/library at hand.  Which is easier than any other language that I work with where I have to learn the philosophy of every package I use. (or collection in the case of the tidyverse on R).  In addition, the tooling was easier. Visual Studio Code has the Calva extension, which makes working with Clojure projects automatic (pretty much anything on the Java virtual machine needs an IDE to handle the project setup, so a good IDE is essential.)

For learning logic programming, I started with some Prolog materials, because that would allow me to focus on the logic and thinking part (Prolog is also fairly sparse in syntax).   I got Adventure in Prolog by Dennis Merritt and followed along with implementing the Nani adventure game as well as the geneology exercise that was developed over the entire book.  But I was always going to move to miniKanren, becuase in any conceivable use, I would be integrating logic programming into something else.

My first two attempts to moving from Prolog to a programming language were with Julia and Clojure.  With Julia, there was Julog (which is attempt to follow Prolog patterns but in the Julia language). This seemed servicable, although all I did was the adventure game. Then I looked at the miniKanren projects.  All of them were the beginnings of an implementation, but not complete enough to do anythihng.  (Scheme and miniKanren both have a reputation for being the target of a budding language creator's first target because they are so simple to write, but then the said creator's attention goes somewhere else).  And even though I have also used Julia in the past, I basically had to learn it over again as it changes every version (I review books by computer publishers, so I have had a chance to look at Julia every now and then, and it does feel like I'm starting over again every time).

Clojure has the advantage the the main language is very stable (and since it is a Lisp it has the advantage of having seen the history of language decisions, good and bad).  They have a fun graphic where the show the history of the source code changing which looks like layers instead of comparable graphics for other language projects that look like landslides.  But the same cannot be said about core.logic.  When core.logic first came out it was a unique in the sense that it was an implementation of logic programming that was in a relatively mainstream computing environment (because logic programming makes a lot more sense on a Lisp type programming environment than on a Algol type object oriented/procedural programming environment).  So there are a lot of early tutorials. But around version 0.8.5 or so there was a major change in the core.logic library organization, and a sub library was created to hold all of the non-logic things. Which includes things like facts and data.  But this broke all of the tutorials. And like faddish things, noone updated their tutorials. So all of the tutorials that everyone points to was from 0.7.6 or so. So as I repeated the Adventure in Prolog exercises, the getting started introduction was easy, but I had to discover that there was a new way of doing things that involved actual data (as opposed to being logic exercises) and I redid the Nani adventure and the bird expert system using the new core.logic and core.logic.pldb structure.  

The bird expert system exercise was particularly difficult. I actually did not do this set of exercises when I went through the Adventure in Prolog book (because it did not actually start until about halfway through).  So I tried to start from someone else's Prolog solution.  And that completely failed.  So I used OpenAI's ChatGPT and Google Gemini to help me. So neither of them completely got it right, but they got me on the right track. So my solution does not look anything like the Prolog solution. And the types of mistakes that the Gen AI did were interesting.

Generative AI works by going through the training data (essentially the internet), and using the tokens (roughly a word, sometime part of a word and sometimes a phrase) in the query, identifies other uses of that set of tokens and comes up with a probability of options for the next token.  Then chooses the next token randomly based on the calculated probabilities. Then, including the token the Gen AI just added, repeats the same and get the next token. And repeats.  The randomness is what gives Gen AI its creativity instead of just being a search engine. But it also leads to mistakes, as the Gen AI does not actually understand any of its source texts, so it does not recognize the context of its sources or the fact that some sources may not actually go with others.

This gets more problamatic in a subject like core.logic, where the majority of the texts on the internet are out of date, in a breaking way. Normally I say that Gen AI is particularly good at computing related topics, but that is because of the vast quantity of material available on various message boards programmers and computing professionals frequent to ask questions and get them answered.  Clojure core.logic is very different, as there is not much material (Clojure is not one of the more common languages, and logic programming is also a small niche), and there are at least three different eras, which are not mutually compatable.  And since modern examples do not overwhelm historical ones in quantity, things get mixed together. 

Now, how big of a problem is this.  In my experiences using Generative AI to aid in programming (again, I am a data scientist, so I am interested in data type issues), Generative AI is good for giving programming structure and style (which is very useful, (re-)learning new APIs is time consuming), but it regularly gets logic and the model wrong. But as a scientist, logic and the model are things I am good at, so I don't mind examining code to correct the logic and model, I wanted the help in getting the thing into a running state!  This is why despite Microsoft reporting 40% error rates in Copilot generated code and OpenAI reporting 70% failure in software engineering project when using Gen AI, professional programmers still find Generative AI to be very useful.  It does get things like how to work with an API right, and has pretty good programming style (with appropriate commenting!)  But logic, which the Gen AI gets wrong, is something that any competent programmer does not mind doing themselves.

The key for using Generative AI is the same as other things. It is good for style and structure. Not so good for facts and logic. But that is what subject matter experts are good at. (and most subject matter experts are not so good at style and structure)  So a trained SME can play to a Gen AI strengths and deal with the weaknesses. But only if the human is paying attention to this. 

Next steps, repeating the Adventure in Prolog exercise, but using the Kanren library in Python,

Why take opportunities for public speaking as an analytics professional

For many of us in technical fields, public speaking often feels like a skill we left behind in school or perhaps dusted off for job interviews, especially if our roles involved training or teaching. Once we're in the professional world, the focus tends to shift solely to our day-to-day tasks, and public speaking opportunities seem to dwindle. However, effective communication is crucial for professional growth, and unfortunately, workplaces don't always provide sufficient feedback on technical presentations.

This is where engaging with local professional communities can be incredibly valuable. While I've had the privilege of speaking at professional society conferences, I've also found immense benefit in giving talks within local technical organizations. Many metropolitan areas are familiar with these as "Meetups," named after the platform that serves as their online home. These local speaking engagements offer distinct advantages compared to academic talks or large industry conferences.

Low-Stakes Practice Environment

---

One significant benefit of giving technical talks locally is the opportunity for low-stakes public speaking practice. These communities are typically smaller, comprising individuals genuinely interested in professional development. Because many members also use these meetings as a platform to share their own insights, the environment is inherently supportive and sympathetic. It's a space free from the competitiveness that can sometimes arise when individuals are trying to build a reputation or feel they're in direct competition. This fosters a very friendly atmosphere for honing your presentation skills, where attendees genuinely want to see you succeed.

Sharpening Your Communication

---

Secondly, preparing a talk for a public audience compels you to think critically about what truly matters. In a work setting, it's easy to gloss over foundational concepts because everyone involved in a project is assumed to have that background. In a public forum, you're required to identify the essential information and ensure you cover it as necessary background. This is particularly true for work-related topics when you might need to use public datasets (as most companies don't permit the use of proprietary data for more informal talks). This process forces you to consider what's important for your audience and what's technically crucial. It's an excellent exercise in organizing your thoughts and effectively communicating them, a skill that translates seamlessly back to your work when you realize not everyone on your team has the same background knowledge.

Building Professional Community

---

Finally, these local groups are instrumental in fostering community. Recent articles in local Pittsburgh publications have highlighted the increasing difficulty of forming social connections after school, and professional colleagues, while valuable, often don't entirely fill this gap due to shorter average tenures at companies and the inherent limitations of work-only relationships. Professional organizations offer the unique advantage of being specific enough to align with shared interests, yet broad enough to expose you to ideas beyond your immediate work. Giving a talk provides a natural reason for others to engage with you, sparking discussions and building relationships that can extend far beyond any single job.