NYC Personalization Pioneers Summit – APMEX

Read the full transcript

[Andy Mueller] Okay, so, first thing’s first, what is You know, why am I talking to you? What are the challenges we’re trying to really use Dynamic Yield for? And, how do we find our way through the data through trial and error to use Dynamic Yield? So, was already introduced, is really the world’s largest online buy in and collectable coin dealer. It’s a niche market, but if you’re interested, you may have heard it. So, we do about 300 monthly sessions, over a billion in yearly sales, we have web and phone channels. It’s a two-way marketplace, meaning people can buy and sell on our platform. And we’re operated out of Oklahoma City in an old Federal Reserve building. So, the important thing to note when I’m talking about, is although we’re kind of a niche market, kind of a weird retail and finance fusion, we really wanna be best in class when it comes to the traditional E-commerce experience. So, category structures, PDPs, Add to Cart, one-page checkout, acquisition teams, retention teams, email marketing teams, all things that you’re dealing with as maybe a retailer, that’s what we’re dealing with, just with a weird product. So part, obviously, part of trying to be best in class, and not compare ourselves to, you know, people in our domain, but also you know, the Amazons of the world, the Urbans of the world, is we need really sophisticated understanding around our segments, around our personalization techniques, and our targeting. I came into the company a couple years ago, and that was really a core focus of what we were trying to do from a data perspective, and I immediately found a lot of difficulties. A lot of these you can probably relate with, so I don’t wanna stand up here and complain about how hard my job is, because all of your jobs are hard, but I do think we have some interesting dynamics at play. So, we have a really diverse customer base, and you were already introduced up here to one, a coin collector, crazy coin collector. So we have investors, core unit of who we’re selling to, hard-core, financially savvy individuals looking to diversify their portfolio, know the market, know the prices, they wanna come in, they wanna get out, they wanna get the best price for the most metal possible. We have this concept of a collector, totally different, totally different. Someone who wants to complete a collection, someone who is an avid collector, someone who understands the right year, and mint, and grade for a coin. They might be looking for one coin out of all coins on the earth, and we might have it, and we need to find that person, we need to get that person to that coin. And then third, kind of most humorously, our beloved preppers, you know, the people that are sensitive to the new cycles, people that are in their bunkers, waiting for the shit to hit the fan, waiting for the next Trump tweet to act on whatever impulse to buy silver at lower volume. So, the important thing here is that the afinity is really tough, right? So, it’s not like, say, you know, where, maybe you’re into kayaking, and you’re into mountain climbing, and I can serve you that, right? We’ve gotta get to really the motivation, and the means, really, really quickly. So, do you have the money to spend? Are you interested in investing? Are you a collector? If I whiff on that, then that’s probably the only chance I get, right? They’re not gonna like, dig into the catalog, because they’re thinking, “I’m on the wrong site, “this isn’t where I wanna be.” “I’m looking for one coin, and they’re trying to sell me “tens of thousands of dollars worth of gold.” Right? So, it’s a really niche industry, completely unique, very little margins, very logistic-driven, very, it’s an ancient industry we’re trying to cram into an E-commerce platform, which is… I won’t even get into all that, it’s an interesting dynamic, right? Large product catalog, we have, at any given time, over 13 thousand SKUs on the site. And the fun part is, they all look basically the same. So,so, this is an actual example. So, you have a coin here, a five dollar and 72 cent, one-fourth ounce silver coin from Armenia with a stamp of Noah’s ark, okay? Somewhere out there, someone’s trying to find that coin. If it’s you, come talk to me, we can place an order. Another item pulled form our website right next to it, a 44 thousand dollar gold bar. So that’s basically twice as much as the car I drive. So like, what do you do, right? Well, we’ll get to that. And then, the last really fun piece that is difficult, but we manage, it’s a really time-sensitive market, right? So imagine if all of the SKUs you sell on your website don’t have a fixed price. Imagine that they are, every five seconds, changing in price, imagine what that would do to your creative, imagine what that would do to your processes. And then imagine that something happens across the world, and it automatically either tanks or spikes the demand for your product. Great, right? So, we thought personalization segmentation was the answer, but we thought no one could help us. We thought this is too niche, there’s too much going on, there’s too many moving parts, so what did we try to do? We tried to build a jalopy, right? So we tried to do it ourselves, and we had some success doing it ourselves. And a lot of you have done this, you know how hard it is to try to cobble together your own data sources, and do your own CRM, and build models on top of it, so that was kind of a core, to this day, capacity of my job at APMEX is managing data science and analytics teams, building customer scoring models, persona models, classification models on our products, purchase prediction models, all this stuff that comes from all these sources, right? Our transactional data, and our SQL servers, our data from GA360, premium products, our data from email partners, our data from AdWords from some display, all of that finding it’s way into something like an actual email you have here on the right, from 2016 where we thought we were pretty hot, we weren’t, because if anyone’s done this, you know that two things become pretty abundantly clear, it’s super time-consuming, it’s super laborious, right? You have to have a team to do it, you have to have a team that usually is paid pretty handsomely, it takes time, they get poached a lot, because data scientist analysts are expensive. So that’s one thing that you realize really quickly, and the other thing that you realize is it can’t scale. Yeah, it’s one email, great, and it did well, great. What next? What about our other channels? What about phone? What about the next email? What about the website? We did not have an answer, right? But we did have a lot of good data. We had a lot of great data, and we knew where we wanted to go, and that’s really where Dynamic Yield comes in. Lots of reasons to choose them, we ultimately loved two things, for the purposes of this presentation, in our day-to-day lives so that makes a really robust CRM solution, which, based on what Liad had to say is getting that much better. So not great following him, when it comes to talking to the CRM, but you can kinda see where, what we’re doing with it. And the second thing that we love about it, really from the data perspective, is that they were very flexible about giving us integrations with different ways to really judge our own lift. So, David, our Account Manager, has been great, from the very beginning it was always kind of a humorous, fun, banter, bring your data with you, you know? We’ll compete, we’ll bake-off, fuse with our data, you know? There was never any concept of, that’s yours, that’s done, you’re with Dynamic Yield now, it was, let’s make this better together. So real quick, I’m just gonna talk about our CRM solution, how we’re really nesting targeting together with Dynamic Yield, and with our own CRM, all the data that we worked on beforehand. We’re gonna do a live demo, if the WiFi cooperates, and then we’ll talk briefly about measuring lift. So, I can’t take credit for this, but this is the way we think about our CRM data. So this is an actual image from Dynamic Yield, you may have seen it, and it really is a great representation of what, how we think about our data into three groups, right? So you have above the water, you have the browse data, you have all this data that is on the cookie that Liad was just talking about, right? The stuff that is super reactionary, super quick, real-time, powered by the UI, stuff we could never get at before, you know? Stuff like you saw in the Urban demo. There’s no way that we could ever produce a script or a function to react that quickly. Super, super valuable stuff, but DY will be the first to tell you, that’s not all, right? That’s not everything that we have, right? We have all of this other data, we’ve been in business for 15 years. We’re a dinosaur, when it comes to E-commerce, right? So we have all this CRM data below the water. Transactional data, all the basics. 15-year order history, 15-year sales history, your lifetime value, date of acquisition, what channel you were acquired from, are you a dealer? Do you sell to us? All of these just, you know, basic transactional data points that we’ve gathered over 15 years. And that’s really great, deeper than that, and what we really wanna focus on, is that domain expertise that we can bring to the table, right? So, you know, David is not, Dynamic Yield is not the coin dealers, right? They are a platform, and they have enabled us to inject our models based on our domain expertise into a platform that’s super flexible. So, we’ll get into kinda how we do that, but kinda like what we talked about with the jalopy. Custom attributes, segment scores, persona, produced by an internal data science team, driven by inside, my internal data science team. So, really simple example that isn’t rocking the boat, but I do think it’s a great example of just a basic CRM integration that we do, is a loyalty program. This was talked about some in the Urban talk, so all sorts of data points here that DY doesn’t have, you know? We have a loyalty program based on different tiers: have you bought this much? Are you a dealer? Do you have a relationship with a sales person? Have you accepted our offer? All of these offline data points that we can then pivot on when you’re on the website. So you can see real clearly in the sorry, in the back, we have real simple audiences based on the CRM feat. Are you a gold member? Are you a platinum member? Are you a silver member? Which I always thought was kind of confusing, since we sell gold, silver, platinum, but that’s not really my area, so. Really, really simple application that, frankly isn’t that easy to do if you don’t have a solution that reads a CRM, offline data get ingested to react dynamically to have pop-ups, to have suggestions to maybe, make another purchase to get you to your next tier, or to have the actual name of their sales representative from the phone channel that they might be buying from. So, really, really simple, but really, really powerful, and stuff that we weren’t able to do. Getting into the more fun stuff, stuff like customer scores. So again, we’ve got phone data, we’ve got multiple channels that DY doesn’t have access to, we’re able to upload that. So it’s not just the customer’s score, it’s based on all of these offline and online, you know, and really 15-year order history is based on how often you’re buying, how frequently you’re buying, the volume you’re buying at, how much money we’re making on those buys. Also, stuff that DY does have, sessions, you know, integration with our ESP, emails, we’re able to turn that on our side, and push it up to Dynamic Yield, because our company understands it, we understand what the customer score is, we like the customer score, we don’t want to get rid of the customer score, we wanna use it. So again, you can see very simply, the CRM audience here, that Yusuf is gonna show you in real time, really easy to build, really easy to activate. And then something that is more and more advanced, our buyer personas. So this is a hypothetical example. So, say a customer 123 comes to the website, right? Maybe they haven’t bought in a while. Maybe, you know, we’ve only been using DY for a year, maybe they haven’t been on the website in two years, you know? But maybe they’re coming to make a ten thousand dollar purchase, a hundred thousand dollar purchase. So I know, because of my CRM, and because of the work that we’ve done, that their customer score is seven, high value, I know that they’ve been buying for four years. Based on persona modeling that we’ve done, I know that you’ve got a 97% propensity for silver, and I know that you’ve only bought investment products. So at this point I know you’re ready to spend, you have spent, you’re a long-time customer, and I know in that first page load, okay? So maybe I didn’t know you, but because of the CRM, and because of that acceptance of that data, I’m not gonna wait, I’m not gonna wait to guess what you might wanna buy, I’m not gonna show you a five dollar Noah’s ark coin, I’m gonna show you 48 thousand dollars worth of gold bars. And we’re gonna try to get you to convert on that. And again, DY is amazing for this, right? So they will build it on the fly, we’re gonna see some of that, you saw that with Urban. But, they don’t always know, right? You do lose that cookie data, you do, you know, you have to get authenticated, and logged in on the site for us to know this, but once we do, we use DY to execute on that strategy. If we don’t have it, then we’re gonna let DY take over, which is the majority of our traffic. So, really what we’re trying to do, is we’re trying to flip the iceberg. We’re trying to say, because of the CRM, and because of the willingness of DY to integrate with our data structures, we wanna target with our CRM, the version of the customer that we know works, but then we want to basically tier that down. And the reality of what happens is, actually in the platform, we’re able to put our best foot forward, based on internal model CRM data, but as people don’t fall into those groups, DY takes over. And it’s looking at the actual in-session data, to then really help the experience. So the reality is, most of the data is down here. Most of the new users, different device users, unknown users are getting real-time personalization, real-time personas built for them. But, we wanna make sure that we’re giving that, if we know you’re a long-time 15-year repeat customer, that you’re getting a really, really similar feel every time, we’re able to lock you in at the top of the funnel, or at the top of the nested targeting. And with that being said, hopefully we can again, WiFi has been a little finicky, walk through how this actually happens on the website, in real time.

– Absolutely, alrighty, so while he’s getting us set up, I’m gonna speak a little bit to kind of what Andy talked about, and so, you know, we have this rich data at the top, and then we also have some of that engagement layer, and then we also have the vent layers, right? Where, it could be the market moving, and so what we’re trying to do is, first and foremost, we’re trying to take everybody out of that generic pool, and drive them into a category as soon as possible. And just as Andy mentioned, when we’re pulling people out into different segments, we’re seeing conversion rate, not just slowly, incrementally increase, we’re seeing it jump. And that’s what we’re constantly trying to do, is pull people out of that generic bucket, into a grouping with an experience. And so kind of what I’m going to demo for you guys, once they get that set up… Oh, perfect. So, now that we have all of this CRM data really well-executed on our end, we’re using DY. So I’m gonna show you how we then pull that in, and how easy it is. So you’d simply go through, you select out CRM data, you have your value, your name. So you could have metal persona, product persona, you could have all these different things, based in your CRM. You could have even the person who you’re VIP customer is assigned with in the call center. And so that way, on the site like Andy mentioned, it’s actually showing their name. So one of the things that we’re, we use a whole lot on our website, is we’re going in, and we’re segmenting by metal. And I’ll create lots of audiences like this. So let’s say we’re going for that gold buyer, we create an audience, I typically like to have them refresh with each and every session, if not every day, depending on how fast that can change. Because if you come in and make a purchase, we want that next experience, that next email, to then be segmented. And so we’d go in, we’d name it, create it, and by going through and creating a lot of these audiences, we have tons in there, and the reason being, is that, let’s say we haven’t gone and had a ton of ideas to go and segment, now with predictive segmentation, it’s gonna actually tell us, hey, that audience you created back on day one, this would be really great if you served them this type of strategy, versus another one. And so it starts to get even better as we build out these audiences, and we let them run, and then whenever we’re spending time doing our tests throughout the week, it starts to show us those. Now I’m gonna show you in action on our site, what that kind of looks like. So this is one of our most popular sections on the website, it’s on the homepage, and in fact, I wanna say 95% of our homepage is powered by Dynamic Yield. And so it’s a really powerful platform, and we’re customizing as much as we possibly can. It’s a common term in our CRO meeting every week that I’m leading up, to say, “okay, let’s figure out how we’re gonna take this section “over with Dynamic Yield.” And then truly run some tests, and optimize for the future. So, we come into this section, and we’re using the Dynamic Yield, or we can go in and we can start to look at the different experiences we have, and the different Dynamic content modules or recommendations. So within this, you can see, this is a silver top picks buyer, and that’s because Andy’s actually gone through, and he’s an avid silver buyer, so it’s throwing him into that segment. Then, let’s say you have someone who’s a gold buyer, so it’s gonna start to throw them gold right away, as soon as they’re getting to the site. We’ll even do things like throwing them very elite products, or silver products, if they’re simply looking to stack a large amount of shiny silver. So now I’m gonna show you how we start to engage with a customer, and drive a more relevant experience, not just from all that built-up CRM data, but what if they just came to the site? What if we don’t know a lot about them? How can we tie that CRM data, and also that quick engagement that we’re getting on the site, to then drive a more personalized, relevant experience? So I’m gonna hop over here to an incognito window, give a page refresh, alrighty, so here it is, and actually you can see that the market right now is on the move, so that’s pretty interesting. So, typically what we’ll do is we’ll go in and we’ll throw just general top products. So you saw how before, because Andy’s bought silver, it was saying silver top picks and it’s showing him silver, now it’s just precious metals top picks. We’re showing a little bit of most popular gold, the average customer is buying popular silver, so that’s what we’re showing primarily. And this isn’t something that we’re telling Dynamic Yield to do, we’re sending a strategy, and then letting it pick from tens of thousands of SKUs, and decide what a customer would buy. So now we come in, and let’s say that I’m very interested in silver, so I start to click a more popular silver item. I go through some of the images, I’m engaging with the site, I’m looking at prices, looking at related products, and let’s say I decide to add this item to my cart. Then, we go through, and maybe this coin from Great Britain is a little bit more up my alley, so I add this item to cart. And then go through a little bit more, look at some images, and now we’re actually gonna start to pivot Dynamic Yield off of that experience, we’re actually gonna start to look at that customer, what they’re engaging with, as well as their previous data, and start to modify our page based off of what they’re engaging with. And so now you can see we’re throwing silver throughout the page. So let’s say we shift gears, and we have someone who’s interested in gold. So they’re going through, they’re maybe visiting a gold category page, or a grid page. They’re going in and they’re engaging with actual gold items. And so now, we can tell pretty quickly, that this person has shifted gears, and now they’re looking for a gold product. One of my favorite is this gold buffalo, so I’ve clicked into this, I could maybe engage with a video and start to watch it, I could put in a quantity of, let’s say, ten, because I’m a big baller, and I really wanna go all out with gold. So I add that to cart, and now, let’s do one more, just so it really knows that we’re gold. So I’ll go to do this really popular Canadian maple leaf, and now I’m navigating back to the homepage, and I’ll show you that top picks module, as well as some of our banners. So we’ll actually start to go through and modify our entire homepage, as well as other areas within the site, off of that really quick engagement that we’re getting that tells us, hey, this person is interested in gold, or interested in silver, or this certain product line or another. And we actually start to look at things besides just a surface-level of what it is, so we can start to look at things like price. So let’s say someone’s coming in, they’re a really elite shopper, this person got in on bitcoin when it was super low, and now they’re cashing out, they’ve got a ton of money that they just don’t know what to do with, and they decide that instead of buying a new condo in New York, they decide, I wanna buy a quarter of a million dollar penny, or a quarter of a million dollar coin. And so, if you are that person, as Andy mentioned, definitely hit us up, we’d love to get some credit, and if not, get on the site and click on a Dynamic Yield module And so, then let’s say, you know, you click on another one, you’re viewing through, you look at images, we see that you’re not the average customer, right? You’re maybe a quarter of a percent of our customers, you’re looking at very niche products, you’re looking at very high-margin, very rare items, and we’re able to then segment out on the finest detail to some of those customers, and show them more relevant items, and recommending them those hundreds of thousands of dollar coins, that typically the average person would just never look at, or never care to see. And so this concludes the live demo. I hope that gave you a good idea of what the type of things are that you can do. So now I’m gonna kick it back to Andy, to go over how you can actually start to measure those results of all these different experiences and variations.

– Alright so, yeah, that’s,, if you want that quarter of a million dollar penny, that’d be great. So, I’ll kind of blaze through, I know we’re running a little low on time. So what, right? We have the model, we’ve got the CRM data, we’ve got Dynamic Yield data working together, moving the site all around, doesn’t matter. And obviously, the answer is yes. So, the issue is not necessarily does it matter, it’s how do we communicate it? Because all of our systems, you know, our bottom line numbers, Dynamic Yield, Google Analytics, all these things are telling us yes, how do we unify all of it? I won’t spend a lot of time on this, because it’ll get everyone mad in the room, probably. Everyone knows what this is like. New shiny toy, bunch of old shiny toys, how do we get them to play nice together? Probably enough said. The good news, based on my previous slide of why we kind of chose DY, they have no problem integrating, no problem writing custom work to try to integrate, so that’s what we actually did. So we’re a big Google Analytics user, big Google, big Query users, we love that as our method of choice. We actually were able to get the exact experience information in to Google Analytics. So here you see a variation of the top picks that Yusuf was showing you, five percent control, I know it’s really tough to see, I apologize, but we actually are able to mimic that exact view in our GA platform, which is considered sort of the record of truth in our business. So, actually able to see that five percent of sessions went to a do nothing action, or a default no DY action, and we’re actually able to see the lift in GA, and in a way that is even that much more real. But that’s not enough, right? We also wanted to know, “hey, if we ran the entire homepage on “eight different modules, and we’re running five percent “controls on those modules, “that means no one’s ever gonna not see some DY content.” So we really wanted to do something counterintuitive, which was totally shut off DY. And DY did something very counterintuitive, which was encourage us to not use their software, which is funny. So this is the true wholistic lift that really, for us, is the ultimate, sort of compass, which is five percent of the sessions that come to our site, we have cookie on that session that says do not show this person any personalized content. The other 95% go nuts, like you saw from Yusuf, and we actually, this last quarter, we saw 4.7% increase in conversion rate. We port that data into Tableau, we port all of that data from Google Analytics back to our DY software to build more models, to push back into DY. Really, really excited to see all that stuff on the roadmap about more work when it comes to the CRM, and really be more open about that data. Ultimately, all this leads to, for this last quarter, a 304 dollar AOV lift, a conversion rate jump of 4.7%, and a revenue per session jump of 4.42%. Keep in mind, we’re not your average company, 304 dollars is a lot, but when you’re selling 50,000 dollar gold bars, our AOV is pretty high. Still, massive jump, right? So just think about this for a second, like, Dynamic Yield, the 95% of the traffic in queue one that got Dynamic Yield, we convinced those individuals to literally buy precious metal at a 4.7% increase over the five percent. Like, we convinced people at 4.7% higher rate to give us their money for a physical precious metal product, to trust us, to really, you know, invest with us, like this is not a normal transaction, right? This is a lot of money, and it’s a lot of trust, and it’s just really amazing to us when we put it in that context, that we’re really increasing our pool, and increasing our market share of a really difficult space. So really briefly, where are we going next? This is a screenshot of our ESP of LISTRAK. You can kind of just see all the same attributes that we use to drive Dynamic Yield, they’re also the exact same attributes we’re using inside of our email segmentation. This is an actual personalized list for an individual from our website, we’re able to pair hashed emails anonymous, you know, no PII, we’re able to connect systems, we’re able to send emails to segment the DY reads, we’re able to push people the emails from DY, it really drives down on the time it takes to execute on email. And really more importantly, there’s no way that a human being, a merchant, can pick a set of six products for one person out of 13,000 SKUs. So we’re really, almost a 100% reliant on the site and merchandising, email merchandising capabilities of Dynamic Yield at the individual user level, and it’s not that hard. I mean, they’ve made it very easy. If you’ve got a CRM in another system, and it accepts the hashed email value, then DY can talk to it, and you can DYFI, whatever that is. We really wanna get more aggressive with prospecting personas, so. Yeah, I know if you’ve bought from me, I know that maybe you’re a gold buyer, but what if I wanna know what to serve you before you’re even in your first page load? And so we’re working with our acquisition team to actually read the content of the ad, and to just start shoving people into our best foot forward, right? That’s kind of our concept is, if you clicked the ad and it was gold, then why am I showing you silver? Silver looks really different, different color, its attention, and much like Yusuf, you’re a big baller, right? You want some gold, and we wanna give you some gold. So we’re really working on that concept. And last but not least, we really wanna tie all the concepts together, and we really view this as the final frontier, this idea of taking events and personas, tying them together, right? So, if I send you an email that says the silver market is on the move, right? And I know that you’re a silver buyer in my email system, and you show up on the site, and I know you clicked that email, and I know you’re a silver buyer, and I know the markets on the move, you’re pretty primed, right? You’re experience should be radically different than a collector. And so we’re really experimenting with really aggressive movement to really transform the look and feel of the site, to something different, because we really feel so confident in our version of you as a customer, and so confident of what’s happening in the market. So again, thanks so much, difficult to follow the CEO of the company, but, we did our best. Thank you so much.

Andy Mueller, Director of Digital Analytic at APMEX, provides a live demonstration of how his team creates real-time personas for different buyers of precious metals and crafts unique experiences for each audience.