Aurelius Podcast – Episode 58 highlights with Ari Zelmanow:

• Being a strategic researcher and what that means
• Learning how businesses work to speak that language and connect UX research to what matters across the organization
• UX Research Democratization and Ari’s thoughts about it
• The role of UX research, how it’s evolving and thoughts about how UX and research can adapt be more valuable

We’re back! It’s been almost 2 years since we launched an episode of the Aurelius podcast and it feels good to bring the show back and have these awesome chats with folks in UX, product and research. Whether you listened to us for years or you’re brand new, welcome!

Our first episode back we have Ari Zelmanow. He’s head of UX research at Twilio and has some very solid chops as a UX researcher having been at places like Twitter, Panasonic and Indeed.

Interestingly he’s a former metropolitan police detective turned UX researcher. We talked about that, his transition into tech and UX research as well as any parallels between being a detective and a ux researcher.

We touched on the topic of democratization and Ari’s opinion on that and why he thinks it’s the wrong debate for our industry to be having in the first place.

As is usual on our show, we talked a lot about learning how businesses work in order to speak that language, connect your work to what matters and ultimately have a greater impact as a UX and research person.

Overall we talked about the concept of being a strategic researcher, how the role of UX researcher is evolving and how we can all adapt to be more valuable as technology and the business landscape changes.

Links from this episode:

• linkedin.com/in/zelmanow
• https://arizelmanow.com/
• https://influentialresearcher.com/

Episode Transcript

(this transcript was automatically created using our very own transcription feature in Aurelius and has been minimally edited, please excuse any typos or weirdness 😀 )

Zack | 00:00

We’re back. It’s been almost two years since we launched an episode of the Aurelius Podcast, and it feels good to bring the show back and have these awesome chats with folks in ux, product and research. Whether you listen to us for years or you’re brand new. Welcome. I’m Zach Naer, host of the podcast and co-founder here at Aurelius. Our first episode back we have Ari Zelman. Now he’s head of UX research at Twilio and has some very solid chops as a UX researcher, having been at places like Twitter, Panasonic, and indeed, interestingly, he’s a former metropolitan police detective, turned to UX researcher. We talked about that, his transition into tech and UX research, as well as any parallels between being a detective and a UX researcher. We touched on the topic of democratization and Ari’s opinion of that and why he thinks it’s the wrong debate for our industry to be having in the first place. As is usual on our show, we talked a lot about learning about how businesses work in order to speak that language, connect your work to what matters, and ultimately have a greater impact as a UX and research person overall. We talked about the concept of being a strategic researcher, how the role of UX researchers evolving and how we can adapt to be more valuable as technology and the business landscape changes. I hope you enjoy it. Let’s jump right into it. Alright. I’m here with Ari Zelman. How’s it going, Ari?

Ari | 01:27

Good, Zach, how are you?

Zack | 01:28

I’m excellent. Excellent. Happy to be restarting the podcast, shaking some of the rust off. We did this pretty frequently for a number of years, and I think it’s been, I have to double check, but I think it’s been two years since we actually did an episode of the podcast. You’re the first one we’re recording coming out of, out of retirement, so to speak. Uh, uh Yeah, I’m excited to chat with you. Appreciate you jumping on.

Ari | 01:51

I am super excited too. Uh, when you and I were talking about the possibility of re reigniting that I was super psyched. ’cause to, to be honest, I’m, I’m just, I’m a, I’m a fan, you know, like I’m a fan of, of, of your work. I’m a fan of your positioning. I’m a fan of like, your advocacy for research as, as a field. Um, and so it was just, just super psyched to be here.

Zack | 02:13

Awesome. Well, yeah, I’m excited to chat with you. Uh, before we jump in, you know, I think it would be really great for you to introduce yourself, talk a little bit about your background for anybody who is gonna be listening to this that maybe doesn’t know who you are, isn’t familiar with your work and your thoughts, uh, just kind give some folks a background of where you’re coming from.

Ari | 02:30

Sure. Um, so my name’s Ari, obviously. Uh, I am the Sherlock Holmes of consumer and market behavior. Um, people wonder how I own that space. It’s because I was a metropolitan police detective turned market detective. Um, so I, if you have ever seen episodes of the Wire or Law and Order or N Y P D Blue, I did that. I did that in a big metropolitan city. Um, I investigated everything from homicides to aggravated assaults, robberies, child abuse, all the, all the different types of stuff. And then went back to school, got my master’s, my doctorate, and then ultimately, um, started, uh, went into the world of organizational psychology. And when I went into the world of organizational psychology, I didn’t love it. Entered the world of market research and just, it just caught, uh, started investigating, uh, like consumer, uh, like foresight and trends, consumer behavior, new product innovation. Really, really loved that. Got bit by the tech bug, ended up building and scaling research teams at some, some pretty reputable companies. Panasonic, uh, a company called Quant, uh, G T M Hub, which it was G t m hub indeed, uh, Twitter right before, uh, it, it turned into what it is today. Um, and I, and now I I I’m on a mission to elevate what insights and research should be in the future. That’s kind of it.

Zack | 03:59

Nice. Uh, there’s a couple things I actually want to ask you about, because I knew that you had a background there, but I don’t think you’ve ever actually told me. What made you have an interest in transitioning from that to UX research? I mean, you talked that you went back to school and you were into psychology, so I can kind of see the, you know, the through lines there, but what was it for you to say? Yeah, I wanna look into this more.

Ari | 04:22

I think I, I’ve, I’ve always had this like, thread of wanting to investigate things. So, uh, another little known fact is when I graduated from college, I was actually a paramedic for a few years for a fire department. Um, and like, I just like following clues wherever they go, whether I like it or not, following clues and then coming up with solutions for those things. And so I think that the transition from detective to like market research, UX research was just a, a love of like understanding, following things, investigating things, talking to people, building a context, and then building a case for something. Um, and I just found kind of a, a, when I found it, it just clicked. It was just instead of investigating like cases that were going to court now, the court is just a little bit different. It’s, uh, business leadership and trying to make a case for why we should do specific things and understanding like, what, I guess I was gonna be really funny about it.

Ari | 05:18

The crimes are of business. Um, I think it just, it’s the, the similarities between detective work and like applied research are just really similar. They’re both really high stakes. There’s a lot, a ton of uncertainty. Um, there’s, there, there’s, the clues can take you everywhere. And what’s funny is people think of, of research in terms of like this deduction, and it’s really abduction that matters. It’s building upon a body of evidence. It’s building a case based upon one thing at a time. Like, you get somewhere and you hear some, you hear a statement, and then you follow the statement and you check the video and, and all of a sudden you’ve got got this like, case of things that point you to a specific direction. Same happens in business. Just, uh, it’s just a really good fit.

Zack | 06:05

Mm-hmm. So this is really interesting because even as you’re describing it, I can start to see, you know, how and where some of the thoughts I’ve seen and heard from you come from. But before we even get into some of that, I really do want to, I have to ask, because you mentioned deduction and abduction in these two worlds. You say there’s actually a lot of similarity if somebody, let’s say, from the UX research or product research space, was curious if there are any parallels in terms of process and approach to how you would’ve done this work as a detective and to how you do it today in UX research teams. Are there, is there any overlap? Is there, is there parallel to that?

Ari | 06:47

How, like, explain a a little more like, are, are you saying like parallels between the way we conduct detective work to the way we conduct research today?

Zack | 06:55

Right.

Ari | 06:56

There’s tons of parallels. If, if done right, the, um, first it’s that you, you get a signal, like you let, let’s, I’m just gonna, I’ll use an example. You could be conducting interviews, right? And you hear something in the first interview, and it’s a signal. And in academic academia, you build on that signal until you hit theoretical saturation. But in detective work, you’re like looking to corroborate or triangulate the things you hear or learn from other people. So I was, um, you, you start with a signal and then you start pulling in other piece of evidence and it’s like a puzzle and you’re putting things together. I think the work is, is kind of the same when you’re thinking about like, doing research on a specific thing, especially when you’re talking about like generative or exploratory work where it’s kind of undefined and you’re, and you’re building something from nothing.

Ari | 07:45

Uh, it, it really aligns. But even without it, it’s really understanding like different pieces of evidence weighing that evidence. So like, this is really interesting. People, I, I’m, I’m become increasingly fascinated with what, with the <laugh>, I’m gonna call it disdain for qualitative research. Like where people are like, well, it’s not as strong and rigorous as quantitative research. Mm-hmm. <affirmative>, well, I posit this. It takes 12 people in a box to convict somebody of murder, 12 people listening to testimony. It’s not big data. It’s listening to testimony from a variety of different sources to decide whether somebody did something or didn’t do something. Mm-hmm. <affirmative>, that’s qualitative research. Mm-hmm. <affirmative>, you’re listening to a bunch of different things. You’re pulling together. Yeah, sure. You’re pulling together data and you’re pulling together other things, but you’re really building a case for something. Um, and I think it’s, it’s about understanding.

Ari | 08:39

So like, the difference between like direct and circumstantial evidence, like direct evidence being something that you actually see and circumstantial being like, well, it’s a part of the circumstances, understanding, like that not all evidence is not created equal. So like, you wouldn’t, it, it’s funny people are are would say like, well, interviews, you know, we, we, we have to be careful about who we listen to, or like product. Man. I had had a really big evolution recently on democratization of, of research for those that have followed me for any length of time, though that at one point I was very pro, then I went very con and now I’m, now I would tell you it’s the wrong conversation to be having. And here’s why. As a detective, you wouldn’t go to a scene and say, oh, I’m not gonna listen to statements from anybody else who observed this that wasn’t a police officer.

Ari | 09:30

But that’s exactly what, when you’re having that democratization argument, you’re saying, you’re saying, well, a product manager conducted interviews, but I’m not gonna listen to what they said because it doesn’t have evidentiary value. And that’s bullshit because it does have evidentiary value. Now, the, the, the researcher or detective, it’s up to them, their skills and abilities to determine the weight that they put on that evidence. Like, oh, well, it was a little biased, so I’m gonna wait it a little less. Or Wow, they asked all the right questions. They, I’ve got transcripts. Like this is clearly like solid, so I’m gonna wait it a little more. But that’s the, that’s what makes researchers, researchers, it’s not gatekeeping all evidence out that nobody believes we, it’s, it’s in court. It’s like the exclusionary rule. You’re gonna exclude anything. I think that’s, that’s crazy.

Zack | 10:18

Yeah. That’s actually, so that’s really interesting. I didn’t expect it to turn into that, but I do wanna pull on that thread a little more because I have definitely read and heard your thoughts on democratization and, and kind of how that changed and, and hearing your reasons why that’s changed. I, I think is like, extremely valid for <laugh> the evidence that you, you brought forward to say, here’s why that’s like, not even the right thing to think about. Because I’ve often said to people that regardless of where a source of data or insight comes from, that there’s something there. It came from somewhere. Now, like you said, even if it’s bias, that’s still information that can be used

Ari | 11:00

A hundred percent.

Zack | 11:00

Does that make sense? And so that’s Yeah,

Ari | 11:02

It’s exactly right.

Zack | 11:04

Yeah. So I, that’s why I’m kind of curious. I mean, then how do you work with that, right? Because I think there’s two questions I have. How do you work with that? Because I do think it should be included for, for more than just purity of research purposes. By the way, I think it should be included, because also, if you’re boxing people out from saying you don’t have skin in the game, to be able to come and capture insights, that politically, organizationally, is gonna put you in a harder position. That’s one reason. But the other is also because it can be used, especially if you recognize it’s bias or, you know, maybe a tainted source or something like that.

Ari | 11:41

Why wouldn’t you want to use all the evidence available? Like, I, it just doesn’t make any sense. And when you’re a detective, you consider things like pertinent positives and pertinent negatives. You’re gonna have a case theory, but a good detective also doesn’t just help figure out what is, but they help kind of figure out what isn’t. Because when something goes to court, a, an attorney has to make a case and say, this is what, this is our theory. This is what we think happened, and here’s why. All the other things, explanations are not the right answer. You can’t get that without talking to others. I think if you think about investigative journalists, detectives, physicians, people that investigate things, it’s their job. The, the power that they have is to go out and like investigate, like, wait things, see if it’s real, see if it’s not.

Ari | 12:28

And just because look, just because somebody like a, I’m trying to think about a, an epic liar that, that isn’t so politically charged that it’s gonna like set everybody off. But even if a, an epic liar says something, it doesn’t mean that it’s a lie. It could also be true. I think the power of a research of research isn’t in the methods that we use or in gatekeeping people into what information. It’s in the ability to make sense of that and build the case and understand it. Um, it’s being advisors to the business counselors, like we should be the ones that are like, okay, cool, that’s evidence. Let’s, let’s put that into the, into the picture and, and figure it out. I thought about this a lot. When you think about the historical context of how this happens. So you have a company, you have product managers, maybe they’ve hired a UX designer, maybe you have some marketers.

Ari | 13:22

They haven’t hired researcher one yet. But to date, product managers are out talking to customers, the CEO’s talking to people. All these people are talking to CU customers. All of a sudden, a researcher comes in and says, Hey, I’m the expert in this. You guys are doing it wrong. Stop. I’m the only one that should be doing this. Like, if you think about how that lands, like, so you’re telling people, it’s just, it’s, it just doesn’t ring. So to me, my, my new stance on this is I’m not having any democratization debates anymore, because I think it’s a stupid debate to begin with. I think the new answer is, any evidence you have for me, bring it my way. I want to see it all. I’ll paint a picture, I’ll wait it appropriately, and then I’ll, I’ll make a case from from all of that evidence. And it’s funny because you think about evidence in the detective world, eyewitness testimony sometimes is not very accurate mm-hmm. <affirmative>. But when it’s corroborated with other things, maybe it’s, maybe it becomes way more accurate. Or maybe somebody sees something, you have something that, uh, that, that totally refutes that well, then you have to make the call. Like it’s, that’s the human element of all of this. It’s why researchers should exist in the future.

Zack | 14:32

Mm-hmm. <affirmative>. And it sounds like you are not of the opinion that the source of information should only be from methods and tactics that a UX researcher employs, or a UX researcher themselves.

Ari | 14:46

Absolutely. Unilaterally opposed to that. I think that the researcher of the future, um, they should be looking at, they should be thinking of it as a layer cake. There’s three LA layers. The first layer that they should start at is what research, um, and tribal knowledge on the issue exists today. Like, what do we understand today? What are the definitions like in detective world? This might be the law and understanding, like the basic stuff, the background. The second layer is what internal data exists, qualitative and quantitative. Maybe you have customer statements from customer success, maybe sales conversations, maybe product analytics, market analytics, uh, csat. People love to rail on N P Ss. It’s a data point. You don’t have to like it, but it’s a data point. I’ll take it. Um, all that, all that. And then finally, those are the bricks. The mortar is the primary research, usually 80, like using Pareto 80, uh, the 80% of the, uh, the impact comes from 20% of the methods. Those methods are usability testing, observation interviews, desk research, and basic surveys out of, out of that. So what is, what data’s available, and that you can start telling a very, very compelling story.

Zack | 16:00

Mm-hmm.

Ari | 16:01

<affirmative>, that’s what researchers should be doing, putting together those sources of data and information.

Zack | 16:06

So the way you just ended that, that’s what researchers should be doing. That was the one thing that I wanted to ask you, is that, you know, if, if, if, uh, anybody in the organization can, and sh and maybe, you know, arguably should be collecting data, gathering research in some way, formal or informal, what does the role then, what is the unique, you we’re talking about business, what’s the unique value prop of a UX researcher to play in all of that? If everybody is gathering data and doing research?

Ari | 16:37

So I’ve been thinking a lot about this recently because I’ve been thinking about, and there’s been a lot of talk about like, this idea of strategic research and does stre str, what do strategic researchers do? And, and how does that play into the equation? And, um, I think when you think about the different roles of research, there are like, um, UX researchers, market researchers, there’s all all different types, different levels, different different groups collecting data. There’s product analytics, there’s all these things. And what I’ve come to the realization is, is that we shouldn’t abolish, like, I don’t wanna say abolish. I, I, I think it becomes a real, real problematic when you say, in every organization, the best structure for research is X. What I do believe is this though, that there is a space, a place for people that do just UX research. There’s probably a space for people that do market research.

Ari | 17:38

There’s a space for people that do strategic research, probably in different companies, and depending on the size of the company and, and the group. Before I get into the next, next level of this though, I, I wanna, I want you to think through this with me. This, this was interesting. Why do companies hire like UX and market researchers? Because when I started thinking about this, it, it hit me that the things that UX and market researchers want to do don’t align with the things and reasons that people are, they’re hired to do that. That, so what I mean by is this, you’re a UX team, you’ve got a bunch of designers, and you’re like, okay, we need to hire a researcher. But you’re not hiring a researcher because you’re like, man, we need a much more strategic thinker here. We are just not strategic enough.

Ari | 18:27

They are hiring somebody because we need to outsource a part of our job so we can focus on other parts of our job. The opportunity costs of doing research are too high. We want to spend more time doing other things, building design, thinking strategically, whatever it is. So they’re outsourcing a part of the job. They’re not like, we are not, we can’t do the strategic stuff. So when researchers come in and they’re like, well, I wanna be more strategic. Well, the designers and marketers are probably like, we got that. I mean, you know what I mean? So like, that’s a problem. But I think that there’s a place for like junior researchers to come in and do more of that tactical level. Like, I’m going to help you with usability testing. I’m going to help you do basic interviews. I’m gonna help you do more design focused research.

Ari | 19:20

Mm-hmm. <affirmative>. And then there’s a place for more strategic researchers. These researchers I am firmly entrenched, should be reporting to the head of strategy for a business, the chief operating officer or the chief executive officer. And what they do is they focus on doing exactly what I described to you. I’m going to look at information across the business. I’m gonna help you build strategic stories based on five things. Growth, value, adaptability, risk and speed. You wanna grow as a business. You wanna increase more value for the business. You wanna be more adaptable to world conditions. You want to mitigate, reduce risk, and you wanna move faster than the competition given things that are happening in the world.

Zack | 19:55

This is a really fascinating way of putting it that I have not heard before. And, you know, throwing my own bias in here, uh, from what I hear so far, I would definitely agree with this because there’s very much a, there’s very much a role and a gap to be filled where we have a thing, we have high confidence in the thing that we made. Maybe we don’t have as much confidence in how we made it. And as you were kind of saying, you know, the role in scale, a lot of junior, uh, researchers could feel is let’s validate that thing. Let’s improve sort of the local maxima of that thing. Yes. Great. That’s value in that work that isn’t, you know, quote, hand wavy strategic work. It’s just not, it’s pretty tactical work. There’s nothing wrong with that. That work needs to happen. There is, on the other hand, there is what are the things <laugh> we should do or make or consider?

Zack | 20:51

And, you know, there was, I can’t remember. I, uh, maybe somebody listening if they want to dig this up, if I don’t find it and put it in the show notes. Uh, but there was a quote by, uh, a, a large study that was done asking executives how they feel, like, how they, how they rate or value the role of insights in their business. It was very, very high. But then they asked the same people, you know, how satisfied are you with your practice of, of insights and gathering today? And it was, it was astronomically low, so huge, you know, huge disparity between the value they placed on it and how satisfied they are with it. And this is where I think the latter half of what you were saying comes in, where there then is researchers, people who create insights that can be feeding information based off of like these much higher level organizational business level goals that then starts to trickle down to define the things that we do, the larger decisions that we make, the tactical stuff comes much later. Am I catching that? Am I summarizing that pretty well?

Ari | 21:54

I think that’s exactly right. And I think it’s, it, it comes from two things. One is there’s the ability to do methods, which is cool. And you could have PhDs who can come in and do methods, but then there’s the language of business and the way that business operates and understanding, like all of those things. Those are more advanced level things that, that aren’t really taught. They’re certainly not taught at, at the junior level. They’re definitely, I mean, even if you learn ’em in school or in M b A program, I mean, you still, there’s still real world application of that. And when you think about, like, when people talk about strategic research, I would bet you that, uh, you’ll get, if you ask a hundred people what strategy is, you’ll get a hundred different answers. And the reality is, is strategy is just helping get, uh, helping provide an intense focus on the things that matter most.

Ari | 22:41

Um, a a researcher is well suited to help be that strategy consigliere. The consigliere being like, uh, what, what in the godfather, Don Corleone, you’re the Tom Hagen, or you wanna be the right hand in the Game of Thrones. Uh, that’s cool too. Um, but you’re the advisor to the business to helping them get that myopic focus on the things that matter most. On the other end of the spectrum, you have these a a junior researcher who could help isolate. Like, here, here’s a very tactical problem. Here’s something that we can help the business move forward, but limited scale and scope. And I think that there’s probably a place for both. Now the challenge becomes this, in a place, in a big, big company, a big Fortune 500 company or a big tech company, sure, you have different departments. You could have market researchers and UX researchers, um, in smaller companies, it becomes, we go back to that fundamental problem.

Ari | 23:35

I’m hiring a researcher to outsource part of my job. It’s the same reason that you hire an agency. I’m outsourcing part of the job. I don’t have time to do 68 interviews, ergo I’m going to hire somebody. Now, I, I, I think that you, that distinction is important. I think the other distinction that you have to make, that I’m still wrestling with is the idea of, okay, if you have UX researchers and market researchers in different areas, how do you make sure that you are laddering up that data to a central point? And that’s where all companies, this is where I’ve been talking about lately, is this strategic insight and foresight team. It’s really a program. You need a governance model to start thinking about that from day one. Because if you’re not, what you’re gonna create is the data version of tech debt. Mm-hmm. <affirmative> the research version of tech debt. You’re gonna have, you’ll eventually grow to a size. We’re gonna have market researchers doing research, UX researchers, doing research, maybe analytics, people doing stuff, all this data, living in different data silos, different places, and nobody aggregating it in one place. That is a problem. An expensive problem.

Zack | 24:42

It’s an expensive problem.

Ari | 24:44

It is an expensive problem. It’s, it slows down the time to decision, um, to fix it. You now have to go back and rework all of these things, and you are now competing against companies that are doing that. And so, look at no further than the world right now with ai, machine learning, natural language across large language models, the influx of companies trying to build the next coolest thing faster than everybody else. Mm-hmm. <affirmative> and speed matters. The, the winner of the data race, the winner of the insights race isn’t in who has more data? It’s in, who can use more data faster?

Zack | 25:17

Yeah. That’s a really interesting way of putting it. Who can do something with it meaningfully?

Ari | 25:23

A hundred percent doesn’t matter. And if you don’t have that thought out, thought ahead of time, or people thinking about it, nobody’s thinking about this. It’s not getting done. It’s, it’s startup 1 0 1. If something’s not assigned to somebody, it’s not happening. So what you end up with is a bunch of the repository problem. You have a bunch of stuff stored in a bunch of places, but nobody like really pulling it together. And the reality is, is when you’re thinking about this, the c e o of these companies are, are, they’re not thinking about like, well, I need to look backward at all the data throughout the business to think about strategic insight and foresight. They’re thinking about what do they have to say to shareholders? What do they have to say to, to investors? Um, how, what, what is next for the business? They’re thinking about from a financial perspective. They’re, they’re, they get focused on the, the, the problems in front of them rather than the things that can be done with the data they have.

Zack | 26:11

It sounds like you’re suggesting the data they have could solve those problems or

Ari | 26:15

Inform It absolutely could, if they were going back to do it. But it, it, it goes back to opportunity costs. So now A C E O has to go back and dig through, Hey, I need all the product analytics stuff. I need all the marketing analytics that we have. Like, and these tools just keep coming out with new and innovative ways to provide new dashboards, new things. Well, it’s making those connections that, that is time consuming and difficult,

Zack | 26:38

Uh, yeah. Making those connections.

Ari | 26:41

A hundred percent.

Zack | 26:42

I, it sounds to me also like that latter, that latter mosts point of the researcher of the future seems well suited to fit that role.

Ari | 26:51

That is exact, their strategy coli area is what I would call ’em. They’re, they are advisors to str business strategy, helping the business focus on the things that matter most from the data within the business or the data that exists in, in the landscape. So it’s not just like, it’s not just the c e o going, okay, I’m gonna look at my product analytics, I’m gonna look at my financial data, I’m gonna look at all of these things and cool, I’ve got an analyst. I’ll have ’em do some, some correlations on these things. It’s also, okay, cool. Uh, what are we gonna do? Like if we see signals of another, another covid, what are we gonna do if we see signals of, of, of what, okay, the markets are showing signals that things are turning again. Okay. There’s some, some, some signs of life. Our businesses are CEOs going, okay, cool. With that information and the information we have available, what changes do we need to make for the future? Like, are are they thinking there should be people thinking about that and people in strategy are, but if a business doesn’t have like a, a chief strategy officer who’s thinking about it.

Zack | 27:55

Yeah. Well, so there’s a, there’s a couple things that have been rolling around in my head as you’ve been talking about this. The first one is really trying to think about how someone’s trying to outsource part of their job as to why they hire really anybody. But UX researchers is no different, right? I think that, so on one end you said time, you’re outsourcing part of that job because you don’t have that time. Makes sense. There is another part of it that I’ve seen where people outsource that part of their job where they realize they lack the quality or experience, right? So maybe as a, a founder or a product manager or a product leader, you’re like, I can go out and talk to customers and I can get a sense for what they need and how we need to fix things and stuff like that. But you get to a point where you, you maybe recognize I can’t do that. To the degree someone with a lot more background to dig in really deeper and help us really, really refine that is,

Ari | 28:47

I would bet those are in the minority. I would bet you like maybe an unpopular opinion. I would bet that most people aren’t thinking that way. They’re, they’re, they’re either thinking like, I’ve been doing this long enough that my intuition, I’m gonna drive my product and my roadmap by intuition. Or, Hey, I could talk to customer. I, I know it’s, I think that what they’re outsourcing is, it’s uncomfortable, but they’re outsourcing the part of the job that they don’t want to do.

Zack | 29:15

Mm-hmm. <affirmative>, like

Ari | 29:16

They, I need more time to think strategically to build my roadmap, to build my, uh, my PRDs to build whatever I’m going to be doing or my design. I wanna spend more time on design or actually like thinking through the problem than conducting the research. ’cause research is time consuming. There’s all, and, and what, what, where research, the, the threat to research is today is if all they’re doing is outsourcing the ability to collect the data and then do a preliminary synthesis of this, tools are gonna be able to do that in a few years.

Zack | 29:45

Mm-hmm. <affirmative> mm-hmm. <affirmative>

Ari | 29:47

Good enough. I’m not saying they will be able to replace a human, but good enough. Not today. I don’t, I I stipulate I don’t think that they’re there today.

Zack | 29:54

Yeah, I would agree. I mean, as somebody who’s actively looking into that for obvious reasons, right? Agree. It’s not replacing any researcher right now. So spoiler alert you, your jobs are safe. However, I think it’s very useful to discuss how it will impact our industry specifically. You know, as things go forward. I don’t think the role of a UX researcher will go away. Certainly not in any time in the foreseeable future, but to your point, and a lot of the, a lot of the case that you’ve been making, you know, up to this point in the conversation, which is even the tactical research that can probably be automated a lot sooner than we’re all gonna expect, right? But

Ari | 30:33

A hundred percent.

Zack | 30:34

But there’s a lot of what happens. And, uh, again, I’m gonna touch on some things that you’ve said that resonated with me as to why this is not such an easy fix to automate or, you know, <laugh> AI’s gonna take over. There’s a lot of context within the business people to people that are not inputs to a machine or a large language model or any of that stuff. Those things can’t necessarily be replicated. Uh, they can’t be, uh, you know, artificially fabricated either to then take a limited data set and say, look at it through that context lens. It’s just not really possible yet.

Ari | 31:10

No, I think I’ll even pull this back to the detective world or to, to legal language. There needs to be a trier of fact. Okay. Meaning that like, yeah, sure, it could spit out information, but you need a human to make, like, let’s at least gut check. Is this even within the realm? Could we, should we corroborate it? Should we triangulate it? I mean, I’m not, I wouldn’t, if I had a million dollars, I’m, I’m not gonna trust something with giving me what I should do with the million dollars without knowing where that advice is coming from or what it means. Like, I wanna know more about what’s in the black box before I, I trust that. And I think that that’s, that’s, I think, I just think that’s an important point is that I don’t think the job is going away either, but I do think that the signals we’re seeing is that the job of researcher is going to change the idea that, um, that we are required to go out and do just primary and secondary research and bring that back to the business, and the business is gonna make decisions. I don’t think that that’s the right angle anymore. I think we shouldn’t be just collecting data or just presenting insights. We should be delivering counsel in a point of view to the business. Um, and I think that that is the trend that, that we’re going to see more in the future.

Zack | 32:23

Mm-hmm. <affirmative>. Yeah. Yeah. And I mean, I, I think that that would actually be a very positive thing for industry. And I would also wager a guess that anybody who’s been doing this long enough would probably agree with you. Yes. That is where, that is the role UX research should fit into a businesses we work for. So the, the big question is what is the gap between where we are today and that, and how do you fill that gap? How do you get, how do you get there?

Ari | 32:52

I think there needs to be some level of, of training. So if you go to LinkedIn today, you will find a shitload of data and information saying like, Hey, uh, interviews or usability tests, or like, they’ll talk about all the methods and like upleveling and all the training. Like there’s bootcamps and skills, and there’s not even research boot camps, but there’s like training for junior or people wanting to transition over or become, get into the space. But there is very limited amount of training for mid to senior, senior level researchers wanting to go to the next level. Mm-hmm. Now, selfishly, I teach a course that, that does that. And so I’m plugging that here real quick. It’s called the influential researcher. But putting that aside for a second, what’s important about that is it teaches people how to build a business case based upon the things that a business cares about.

Ari | 33:42

So in our field, you’ll hear people say, oh, we wanna create delightful experiences. We wanna understand what people really want, what people need to be able to use a product. Cool. Not denying that those things are important. The problem is that when people are making those statements, they’re not explicitly connecting them to something the business caress about, which is growth, value, adaptability, risk, speed. It also, there’s very little training, teaching people how to develop and present a point of view to things without qualifying it, hedging it. Um, there’s a whole bunch of, of on of work out there on like trying to find consensus or work with others, but there’s very little on like principled negotiation or how, how to land, land on something, or how to hold your position if you think you’re right, even if other people think their position is right. Mm-hmm.

Ari | 34:31

<affirmative>. Um, and there’s, there’s all of that ties into that. It’s, it’s about evidence collection, which I think you should do. Taking a detective type approach, I think it’s building a case from that evidence. So it’s like following the facts wherever they go, whether you like it or not. And then there’s negotiating an outcome. And what I mean by negotiating in an outcome isn’t like negotiation like we normally think of at the car dealership where you’re like, oh, I’ll give you $20,000. No one say 18,000 negotiating is just steering around or through something to get to an outcome. And the outcome for us is decisions. We want better decisions. Yeah. Um, and research, historically, we, we played this game of talking about methods and tools, um, and then you’ll even hear some talk about measurement of impact, yet in almost every place, with the exception of where I’m at now and right now is when it’s happening, measurement of impact has always been like this, like flimsy, shitty, I don’t even know what it is, how we measure impact now.

Ari | 35:31

It’s really funny, it hit us a few weeks ago, is there’s five levels. Attention, interest, desire, action, and then the, like, the gold standard, and I’ll tell you attention is are people aware of the research we’re doing? And what’s cool is in Google Docs, there’s actually analytics that show you if people are reading your shit, like, wow. So now you can actually quantify if people are aware of your research interest. There’s also an analytic in there that shows you comments. That’s a pretty good proxy for interest if people are commenting, they’re interested. Mm-hmm. <affirmative> a desire to int integrate into their roadmaps. You could tell that from the types of questions and then action. It’s almost like a lagging indicator, um, where you have to look back and see are they ha has this been integrated into a product roadmap or a marketing plan? And then there’s the gold standard, which everybody wants, which is okay, the research we’ve done can be correlated to increased revenue, decreased churn, some metric.

Ari | 36:28

Right? Problem is, is people don’t do that. Don’t measure it all because they can’t hit the gold standard. So they miss the first four steps that are necessary to get to the fifth step to begin with. This is the problem with research is that we’re all talking about these, like, we’re, we’re talking about the PS and not the stake. You know, like the stake is how do we become consultants in the business? How do we become, like, I talk about this in my class, you can be Bob the bagel guy, or Michelle, the McKinsey consultant, Bob the bagel guy. The only advice he gets asked for is, Hey, what bagels good today? Like, that’s what the CEO wants to know. Mm-hmm. <affirmative>, but Michelle, Michelle gets asked, what, what do we do when the game is on the line? Mm-hmm. <affirmative>, that’s who we wanna be. The the advisor, the counselor. Mm-hmm. <affirmative>, not the BEO guy.

Zack | 37:11

Yeah. Yeah. You know, a lot of what I’ve talked about over the years, over the course of my career for, uh, anybody listening, I’ve also worked in UX and research and product strategy for a little more than 15 years. And the single biggest thing I feel like is missing is exactly what you just said, which is having an understanding of the business so that you can communicate how you fit in and the work that you do fits in. Uh, this focus on the craft is great for people in the craft. Everybody outside of that, nobody cares.

Ari | 37:42

Not even a little bit, even

Zack | 37:43

With, you know, Bob, the bagel guy, nobody caress how they made the bagel they want. No, they want a delicious bagel. <laugh>,

Ari | 37:50

That’s it.

Zack | 37:51

Um, so I would, I mean, I I I very strongly agree. It sounds like you’re suggesting the biggest gap is really UX and research folks understanding businesses better is, is sort of first step to getting there.

Ari | 38:04

I agree. That and, and the way we present insights. So the other thing you described, I want to tie to that because I think it’s important. We historically have focused on output for so long that we’ve stopped focusing on outcome. And so the way we deliver insights is a deck. That’s what we do. Um, that is anybody asking? Is that the best way I could tell you? We, we, we at, I work at Twilio, we don’t present in a deck anymore. We have a special form format I developed call a point of view document. It’s a completely different way of presenting insights. It’s short form. It, it’s, it’s, it is, it aligns with how humans connect with and interpret information today. If you aren’t thinking about those types of research isn’t just about, like you said, the craft or the methods. It’s, there’s a, a process of delivery and trying to get synthesis of information and getting, ultimately catalyzing some sort of action from it. Mm-hmm.

Zack | 39:03

<affirmative>, senior

Ari | 39:04

Researchers need to be thinking about that. We need to be able to turn data into stories, stories into strategies and strategies into outcomes. Not, not like outputs, not decks, not like, and, and it’s, it’s remarkable. Really, even the way that we do business is, is fucked up a little bit. Here’s, let me follow this for a, for a second. Um, 20 years ago, you wanted a video game, right? You and I, we had probably Apple computers. I’d go to the store, I’d buy a box off the shelf and I’d have to bring it home and I’d play my video game. I’d put it in disk drive, probably eight disks or whatever. Okay. But here’s what’s interesting about that. When they developed that game, they had to do a lot of research on the front end, because once it was boxed up, it was over. Mm-hmm.

Ari | 39:47

<affirmative>, it was shipped, done today, software could be sent out. You could, now I know that you can’t actually do this, but you could technically, you could redo the whole UI of a software platform over the weekend and have a, have it shipped on Monday. Yeah. Friday to Monday you could do it. Mm-hmm. <affirmative>, um, you could ship, ship repairs, fixes, bugs, all that stuff can be fixed in real time. Now look at the research process. Historically, research goes from discussion guy, like research objectives to questions, development of instrument. It’s very linear, ultimately ending in a report. Now, um, why, when, when software development sped up, all we did was say, okay, we’re gonna be agile with our process. And so we shortened that same line. It’s the same line, we just do it faster. Mm-hmm.

Zack | 40:39

<affirmative>, mm-hmm. <affirmative>,

Ari | 40:40

That’s not a very detective way to do things. Detectives will build upon a body of evidence over time. So rather than go, we need to do 60 interviews over six weeks, we’ll do six interviews one week at a time, over 10 weeks and report every 10 weeks. I say all of that, and it, it, it’s a long-winded way of getting to the point that research has not evolved with the way that that software development or even product development has evolved. If we don’t make some changes, I mean, we’re gonna continue, we’re gonna continue to run into the same problems we’re running into today, which is a lack of perceived value in the market as evidenced by, uh, disproportionate layoffs in that space.

Zack | 41:19

Hmm. You know, one thing that has been kind of sitting in the back of my mind, I really want to ask you this. I know we only only have a few minutes left, but, you know, given your background and experience as a detective and now as a UX researcher and the, that, that whole perceived value, did you ever experience that as a detective doing research?

Ari | 41:40

Well, it’s interesting. When I left the police world and entered private sector, it was hard to shake that detective, uh, that detective halo. Um, because there’s some perceptions about police officers and detectives in the world. And certainly it took a while for me to get, to get, even, even though I had two advanced degrees, it took a while to shake that. I don’t think people think that detectives, like, I don’t think the detective thing is what’s interesting to people. I, to be honest with you, I would tell you that I think the thing that was most interesting, the thing that probably gave me the most juice, I worked at Twitter and it’s heyday, and once you have a logo like that on your resume, it’s a lot easy. Anybody that thinks it’s not easier with a big logo on your resume to get the next job, it’s full of shit.

Ari | 42:22

Mm-hmm. <affirmative>, um, it is, it just, you see it across, across the board. That’s why you see X, Google, s, Netflix, x, Xang, whatever. Um, that said, I think that I, I, I think that it’s, you need, researchers need to position themselves and by position themselves, we don’t control our positioning, our stakeholders do. And right now we’ve trained stakeholders to accept and accept us in a certain way, which is however you’re showing up at work. And if that’s like, I’m gonna bring you data and insights, you’ll make decisions. That’s how they’ve been conditioned, that’s how they’re going to behave. But if you go to them and you say, Hey, I’ve got a point of view on this, this is what the business should do, let’s talk about it, then they’re gonna treat you that way. And it’s, that’s, that’s kind of, I think the detective thing gave me, gave me that.

Zack | 43:15

Yeah. Well, that’s kind of why I asked that is because there was a lot of different parallels I was drawing in my mind. You know, so you say, uh, as a detective, you gather all of this evidence, you continue to build on a body of evidence until such time where you’re gonna share that, and then some other people are going to make a decision based off of that. The detective doesn’t make the decision.

Ari | 43:36

No. And

Zack | 43:37

I, I translate a lot of that into what’s happening in UX research. There’s people gathering data, sharing insights, but they’re not the ones making the decisions. Now, I really hesitate to ask the question, is that right or wrong? I don’t know that that’s so much important, but, you know, what do you make of that? Is it because there’s a separation of those kind of things in other places in the world? Does that make sense?

Ari | 44:00

I think so. I think what I, how I would res respond to that is that yes, detectives come up with, we’ll follow the facts, and then we build, we’ll say like, um, we believe X happened. This is the person who did it. This is the story. Like, remember, I’m building a story, um, about what, what I think happened. And here’s, I’m not only am I telling you the story, I’m telling you, here’s some reasons that other stories aren’t plausible. And then I present that to a circuit attorney. The circuit attorney says, sounds right, we’re gonna issue charges. And then it goes to court in front of a 12, 12, a jury of their peers. They, the case is made. Uh, they, they’re the triers of fact. Uh, and they, they say, oh, I believe the prosecution’s evidence, or I believe the defense is evidence and the decision is made.

Ari | 44:50

Now why there, that’s such a cool parallel is as a researcher, as a detective, as a strategy detective, strategic consigliere, your job is to build that case and present your case. You’re now the prosecution. The other side is the defense. They’re gonna build their case. And then what’s going to happen is the business is ultimately going, some stakeholders, some leader, hopefully, is going to be making, listen to that, make a determination, and they’re going to move forward. And it’s not about, it’s not a, you can only make your strongest case. The business is gonna have to make a decision anyway. It’s the same thing. It’s either it’s going to end in some verdict and the world’s going to move on. And then your job as a researcher at that point is, okay, the decision’s made. How do we help make this the most successful decision? Even if it’s not what I recommended?

Zack | 45:40

Yeah. You know, not something we talked about. But that last part is a really, really important point of distinction for real professionals is, and, and real leaders frankly, is that maybe you weren’t on board with the direction or decision made, but a real leader and a real team player is gonna say, well, I’m gonna do everything I can to make it successful. You know, in spite of the fact that maybe it wasn’t my first choice.

Ari | 46:03

And so this, this is the, this is the distinction between a, uh, a consultant and or a tactician and a technician, a tactician will make sure their point of view is heard, understood, and, and presented. A technician will just do whatever the business tells ’em to do. A technician goes in and says, what do you want me to do? A tactician says, this is what we should do. Here are the reasons why. Here are the reasons we shouldn’t, here’s what, here’s what it’ll take to make this happen. Let’s go. And then the business will say what we’re gonna do. But the tactician’s going to say, okay, I, I might not agree with this, but I’m going to move forward. Technician’s just gonna do what they’re told. And if all you’re doing this is going back to the UX and thing, or the researchers who are hired to be outsourced, help, this is where you got into this world of sur organizations.

Ari | 46:52

What do you want us to do next? Here’s the roadmap. I need you to test these things. And so they’ll just work. The all they do is work the roadmap, they work the roadmap. Um, and if you’re working the, I’m not saying that there, I, I don’t wanna get into whether there’s value in that or not. I think that there’s lots of problems with that, but I also think that to your point, there are some benefits to that that said, that is a very different world than those that are part of the business helping move either the product strategy, the marketing strategy, or the business strategy forward using strategic insight and foresight.

Zack | 47:25

Makes sense. So listen, Ari, we’ve got a few minutes left and I know we gotta, gotta wrap it up. Gotta be respectful of your time. Something I, I, I like to ask at the end of every one of these episodes, if somebody came in, they didn’t hear everything we said, or I developed temporary amnesia, and I were to ask you what is, you know, one statement, what one takeaway, how you would summarize what we discussed that’s important, uh, to share for people who are listening, how would you answer that?

Ari | 47:54

Strategic. Strategic researchers don’t just deliver data. Strategic researchers. Don’t just deliver insights. Strategic researchers deliver counsel. Be a consigliere.

Zack | 48:07

Alright. All right. All right. I’m done with that. Uh, really appreciate you jumping on taking the time to chat. I’ve enjoyed it. I can go a whole lot more <laugh>. There’s several things that I actually want to, to dig in and chat with you more. But like I said, we gotta be respectful of your time. Um, anything you wanna share with folks before we jump off about your work, your course, anything like that and how folks can find you?

Ari | 48:29

Yeah, thanks Zach. So, yeah, there’s a few ways that you can contact me and get involved. If you’re a, a mid to senior level researcher or somebody who conducts research, I teach a course called the Influential Researcher. You can access that@influentialresearcher.com. Uh, there, i, I give you a free framework to, to sign up, learn about the course. Um, there’s a few versions of that and I can tell you about that. Then you can also find me on LinkedIn under, uh, at Zelman now. Um, my name’s Ari Zelman now, so you could find me there. I’m the only one, um, or ari zelman now.com. Um, always open and willing to talk strategic insight and foresight with anybody.

Zack | 49:07

Awesome. Alright, well, thanks again for joining us and, uh, we’ll see you next time.

Ari | 49:12

Cool. Thank you. Thanks, Zach. I really had a good time.

Zack | 49:14

This podcast is brought to you by Aurelius, the platform designed to help you gather research data, make sense of it fast, and turn it into insights in action. It’s a central place to search and share all of your research data and insights. You can try Aurelius for free with a 30 day trial by going to aurelius lab.com. That’s A U R E L I U S L A b.com. If you enjoyed this episode, give us a review on iTunes and let others know. You’ll find all the links and resources to each episode on the show notes@blog.aureliuslab.com. Catch all new episodes of the Aurelius Podcast almost anywhere you listen to podcasts like iTunes, Spotify, and more. Stay up to date when new episodes come out by signing up for our email updates on our website.