Tony Nash joined Geoffrey Cann in Digital Oil and and Gas podcast to talk about his revenue forecasting and predictive intelligence analytics startup company Complete Intelligence — how does the company solve the problem of terrible forecasts and the lack of context around data?
Geoffrey Cann joined us in QuickHit: 2 Things Oil & Gas Companies Need to Do Right Now to Win Post Pandemic.
This podcast originally appeared at https://digitaloilgas.libsyn.com/159-interview-with-tony-nash-of-complete-intelligence?utm_campaign=interview-with-tony-nash-of-complete-intelligence
Digital Oil and Gas Description
160 – Interview with Tony Nash of Complete Intelligence
Jul 22, 2020
Today’s podcast is an interview with Tony Nash, CEO and founder of Complete Intelligence. Specializing in revenue forecasting and predictive analytics, Complete Intelligence develops artificial intelligence solutions. In this interview, we discuss predictive intelligence analysis, how Complete Intelligence works, and what value these forecasts can generate.
Show Notes
GC: Welcome back to another episode of Digital Oil and Gas. My name is Geoffrey Cann, the host of the podcast. And I’m joined today by Tony Nash, who is the CEO and co-founder of Complete Intelligence. Tony, welcome to the podcast.
TN: Thanks, Geoffrey. It’s good to be here.
GC: You and I met probably a bit of a month ago. We did a short video exchange, and it was so much fun, we agreed that we should probably do something a little more involved, and here we are today. Of course, my interest is how digital innovation and digital strategic toolkit are transforming how the oil and gas world operates.
Your area of interest and expertise, the focus of your startup is in the application of smart technologies in agile budgeting and forecasting and market modeling. And that’s a big area of interest for oil and gas. That’s the reason why I thought you’d be a terrific guest to come on the show today and talk a little bit about that.
TN: Thank you very much.
GC: What’s your background? You were with The Economist, is that right?
TN: I was with The Economist. I led their global research business for a while. And I built what’s called the Custom Research business. It was a small niche business when I joined. It was a pretty sizable revenue by the time I left. Great organization. Had a lot of fun there. I then moved to a company called IHS MarkIt. Information services firm. I led their Asia consulting business. And from there, we started Complete Intelligence. I’ve been in information services off and on for way too long, since the late 90s.
GC: And what’s your education background? Did you start out in computer science or something?
TN: I was a graduate at Texas A&M in business and my grad work was in Boston at a school called The Fletcher School, which is a diplomacy school. So I was trained to be a diplomat, although I’m not very diplomatic at the moment. I have my moments.
Part of the reason I started going down this road is because in grad school, I had a trade economics professor who was amazing, great guy. I started my career after undergrad at a freight forwarder and customs broker. I didn’t have a glamorous first job. I was actually working the night shift in a warehouse at a freight forwarder, receiving exports and typing out airway bills and all that stuff. I got to know the nuts and bolts of world trade pretty specifically and pretty firsthand. I don’t know of any other trade economists who have started the way I have. I look at trade data differently than almost every other economist that I know of. I look at it somewhat skeptically. It’s that skepticism that I realized in grad school with this fantastic professor that my skepticism was an asset. My skepticism was an asset within statistical, mathematical models, within economic discussions and so on, so forth.
I had used it in business before that, but I didn’t think that I necessarily had the ability to apply it in this big world before I had this experience in grad school. So I then took it and I joined The Economist. I kind of conned them into hiring me, which was great, and then within a year or so, I was heading their global research business. From there, we just kind of took off.
GC: What are some of the products out of The Economist? Because I buy the magazine every week. And The Economist publishes an occasional handbook of global statistics, GDP by country and balance of trade and so forth. Were you involved in those kinds of products or were the products you were involved in much more specific to a client or customer requirement?
TN: I wasn’t. A lot of those are extracts from, say, IMF data. That’s part of The Economist publishing, which is a slightly different business to what I was doing. A lot of what I was doing was really applied work with clients. Solving real problems, figuring real things out. Some of this was corporate forecasting, looking at costs, looking at revenues, those sorts of things. Some of this was doing work for example the World Health Organization in places like Cambodia, comparing different treatments for mother-child transmitted HIV.
We had all kinds of cool, different approaches. And from my perspective, we could really play with different methodologies. We could really understand what was working and what wasn’t working. It was a huge sandbox for me. Again, really great smart people. That really started a lot of this kind of true love for me, which is what I’m doing now.
GC: What is the business problem that you saw that was sufficiently vexing that you decided to devote a lifetime in a career to trying to solve?
Because your career builds you to a point and then you say, “You know what? This is the problem I think I’m going to aim to solve.“ And you know what? You may go on to solve other problems, but at that moment, why would you become a founder to go solve something unless it was so big and so vexing, it was worth your time?
TN: I think I became a founder because I underestimated how hard it would be to build a business. Almost every founder will tell you that. When I was with both The Economist and IHS Markit, I had two really consistent feedback points that people gave me.
First is the quality of forecasting within information services, within corporate, say, strategy, finance, forecasting units, is pretty terrible. Most people forecast through, let’s say, a moving average approach. Some of the largest companies in the world will forecast using a moving average. If they are super sophisticated, they’ll use a very small maybe regression model or something like that.
But what mostly happens is one of two things. Either they look at last year’s and add a small percentage. “We’re just gonna have three percent on this year.“ That’s pretty common. The other one is really just a gut feel like, “I really think it’s going to be X this year.“ If a Wall Street analyst understood how unscientific the way outlooks are done within large companies, they’d be pretty shocked.
I mean, there is a belief that there is a lot that goes into the sausage machine. Traditional forecasting is terrible. Any forecast you buy off the shelf? Pretty terrible. Any forecast you’d get within a company? Pretty bad. Even the data scientists that are on staff with a lot of these big companies, really brilliant people, but they’re not necessarily fine tuning their forecasts based on error. And this is the key.
Companies who forecast should be required to disclose their error for every forecast they’ve done historically. That’s what we do for our clients. Because the number one problem was the quality of forecasts. So we spent our first two and a half years focused on that problem. We continue our approach to that.
GC: When you say “publish the error,” do you mean error in hindsight? How bad were we last year or do you mean here’s what we think our forecasted error is likely to be this year?
TN: Every year, any forecaster on planet Earth should say, this is what we forecast last year and this was our error rate. When we look at consensus forecast, for example, for energy like crude oil, natural gas, industrial metals, consensus error rates are typically double digits. Typically double digits. We just did a calculation. When I talk about error rate, I’m talking about absolute percent error. I’m not talking about gaming off pluses and minuses because that’s really convenient. But you look at a plus 10, you look at a minus eight, and that becomes a nine instead of a one.
People who forecast should be required to publish their error rates. Companies, especially energy companies, are paying hundreds of thousand dollars, if not seven figures to buy data. Those guys [forecasters] know they’re between 15 and 30 percent off in their forecasts regularly. Businesses are making decisions based on these data.
That’s the thing that, as someone who’s run businesses, not just analysts businesses, but run real proper businesses in different spaces, seeing planning people make decisions with a 30 percent error rate or 50 percent error or whatever it is, but no accountability from the information services provider? That’s a problem.
That’s a 1990s business model where you could play with the opacity around data. But in 2020, that should not be the case at all. We regularly show our prospects and our clients our error rates because they deserve it. They deserve understanding what our error rates are line by line.
GC: In oil and gas, when I’m building up a forecast, particularly for, say, an oil project, I’m having to forecast currency exchange rates, interest rates for my borrowings, the price of certain critical commodities like cement and steel. I’m having to forecast project delivery timeline and schedule. I’m having to forecast future market demand like, where’s my product likely to go? If each of these has a 15 to 30 percent error rate built into them and I’ve added them all up to get to a :here’s my forecasted economics for the year.“ Have I built in and basically had an accumulated error rate that makes my forecast pretty unreliable at that stage? Or these different errors, all sort of stand alone?
TN: That’s the budgeting process.
GC: I’ve been in that process. Right.
TN: Anybody who’s worked on a budget like that, they understand it. Maybe they don’t want to admit it, but we talk to people all the time who tell us. We have a client in Europe who admitted to us that some of their core materials that they buy, they know internally that their forecasts typically have a 30 percent error. And when we say that to people, to other companies, that’s feedback we get consistently that the people who actually know, the data know that their companies have error rates that are 20 to 30 percent or in some cases worse. They’re that far off.
When you think about it from a finance perspective, you’re over allocating resources for the procurement of something and that resource could have been used for something else. That’s one of the reasons why it’s really important for us to help people really narrow that down.
We check ourselves all the time and we looked at some industrial metals and energy stuff based on a June 2019 forecast for the following twelve months through the COVID period, comparing some consensus forecasts and our forecast. On average, we were 9.4 percent better than consensus. This is a Complete Intelligence forecast. It’s an aggregate looking at one of our manufacturing clients.
When you look at the different horizons, we look every three months, what was the error every three months, even up to the COVID period. On average, we were 9.4 percent better on a MAPE (Mean Absolute Percent Error) basis. If you’re buying off the shelf forecasts from some of the typical service providers, you’re looking at a pretty large disadvantage. They’re not using machine learning. They’re not using artificial intelligence. If they are, it’s typically very, very simple.
Now, part of what we’ve done through the process is we’ve removed the human process, human involvement in every aspect of data and forecast. From the data sourcing to the validation to anomaly detection to processing, to forecasting, we do not have human analysts who are looking at that and going, “that just doesn’t look right.“
GC: OK. It’s all done by machine?
TN: Right. We have machines that apply the same rules across assets. Because if we have human beings who gut check things, it just inserts bias and error through the whole process. And with no human intervention, we have a massive scale in terms what we do. We forecast about 1.1 million items every two weeks. Our forecast cycles are every two weeks. And we do it very, very quickly.
GC: And nine percent less error rate or a lot lower error?
TN: For the ones that we checked for that one client, yes. I would say in general, that’s probably about generally right. In some cases, it’s better.
GC: So a few things. One is the huge range of things that you can forecast when you remove all the humans out of it gives you these scale-ups. And then the fact that you can do it over and over and over again in much tighter cycle times than someone who just does it annually, once for a budget. And third, you’re testing your accuracy constantly to improve your algorithm so that you’re getting better and better and better over time.
TN: Exactly. When you consider something like crude oil, there are hundreds of crude forecasters who know that, they know that they know the six things that drive the crude oil price, right? And I guarantee you those crude oil forecasters who know what they know, what they know what those six things are, manually change their output once their models run. I guarantee you.
GC: I remember working for an oil company in Canada where the coming of the oil sands, but it was the monthly oil sands production expectation and would come into the finance function, where I was working, and the numbers would come in the spreadsheet and the finance people go, “add five percent to that.” Because they would say, they’re wrong every month, we’re tired of being embarrassed about being wrong. And they’re wrong because they undersell their performance. So just add five percent. And that was the number that would go to the market.
TN: And then that’s the error, right?
GC: And was that even the final error? There may or may not be on top of that?
TN: Probably not. And there are very few companies, we have some German clients, so they’re pretty good about doing this. But there are very few companies who actually track their error. And so most companies Are not even aware of how far off they are, which is a problem.
Here’s the second problem. The first one is forecasting quality is terrible. So we’ve developed a fully automated process. We measure our error, that sort of stuff. The second one is the context of the forecast. What I mean by that is, let’s say you’re making a specific chemical. You can go to some of these professional chemical forecasters, but they’re not making the chemical exactly where you make it. They don’t have the proportion of feedstocks that you have. Because we’ve built this highly iterative forecast engine that does hundreds of millions of calculations with every run, we can take a bill of material for that microphone in front of you or a chemical or a car, And we can forecast out the cost of every component to that every month for the next 12 to 24 months.
GC: Really? So, at any scale or any, I mean, you do it for a phone, you can do it for a car?
TN: That’s right. So if you look at a bill of material with, say, a thousand levels in it. Not a thousand components. But, you know, if you look at the parent-child relationship within a bill of materials, these things get really sophisticated really quickly. Some of the largest manufacturers have this data. They have access to it and we can tap into that to help them understand their costs, the likely trajectory of their costs over time. What does that help them? Helps them budget more accurately. It helps them negotiate with their vendors more accurately.
If you’re a, let’s say you’re a 20-billion dollar company and you have one percent on your cogs, how additive is that to your valuation if you’re trading at 15 or 20 times EBITDA.
GC: Yeah. And just right to the bottom line at that level.
TN: Exactly. This is what we’re finding. For the high context as of the second kind of business problem that we’re solving, and so we do this on the cost side. We do this on the revenue side. For that second problem, which is high context, again, the platform that we’ve built allows the scale, because if we had analysts sitting there scratching their head, rubbing their beard for every single thing we’re forecasting, there’s no way we could do this scale.
But because it’s automated, because it’s scalable, we can actually do this. And so it adds a whole level of capability within major manufacturing clients and it adds a whole level of risk protection or error mitigation to those guys as well.
GC: Just think about the current year that we’re in, which would include, at least in Canada, a pipeline constraints and the potential for rail expansion activity south of the border to either curtail production, the behavior of OPEC. When you think about getting into forecasting world of commodity prices… I can understand a manufacturer bill of materials and get into cost of goods sold and forecasting quite precisely what their forward manufacturing cycle will look like. I can use the same thing, though, in the oil industry, though, and probably gas, too, I would suspect.
TN: Yeah, yeah. Absolutely.
GC: And what’s the industry’s reaction to it? Because there’ll be people inside oil and gas who are doing forecasting today and they’ll be fairly proud of the models that they built that delivering a forecast. You’re walking in and saying ”I’ve got a whole new way to do this that is so many more cycles faster than what you can manually do, looking at many more products than you practically can. And if I show you that you’re nine percent off, 10 percent off with it.” I can imagine a negative reaction to this. I can also imagine for some organizations, pretty positive reaction on balance. How companies react when you told I can sharpen up your numbers?
TN: OK. So I’ll tell you a story about a gas trader. October of 2018, we went into a natural gas trader here in Houston. We showed them what we do. Gave a demo, give them access for a couple weeks so they could poke around. And we went back to them later and they said, “Look, you are showing a like a 30, 31 percent decline in the price of natural gas over the next 6 months. There’s no way that’s going to happen. So thanks, but no thanks.”
GC: This was your data telling them? All right. Refresh my memory. What was going on in October of 2018?
TN: Nothing yet. But Henry Hub prices fell by forty one percent within six months. So these guys were completely unprepared. The kind of conventional wisdom around natural gas prices at that time were unprepared for that magnitude of fall. But we were showing that that was going to happen. And so when you look at that, we had an 11 percent error rate at that point, which seems kind of high. But conventional wisdom was a 30 percent error rate.
We don’t expect to be the single go to source when we first go into a client. That’s not our thought. We know we’re a new vendor. We know we’re offering a different point of view. But we’re in a period of history where you have to think the unthinkable. And this is 2018, ‘19.
With the volatility that we’re seeing in markets, you really have to be thinking the unthinkable, at least as a part of your possibility set. It’s really hard. I would think it to be really hard for really anybody who’s trading any magnitude of oil and gas product to put something like this outside of their arsenal of strategic toolkit that they use.
GC: Well, certainly, if you had that gap in expectation of gas prices, the gas producer should have been thinking about hedging at that moment. And if their conclusion was, you’re completely wrong and I’m not going to bother with hedging, then shame on them really, because they should have done a far better job of managing to the curve. That’s a great story because it illustrates the challenge.
TN: That’s normal. It’s kind of the “not invented here” approach. And I see a lot of that within oil and gas.
We see a bit more interest in chemicals. They have to understand the price of their feedstocks. They have to understand their revenues better. And so we see a bit more on the downstream where there is a lot more interest. But midstream, upstream, it’s just not really there.
GC: What’s the untapped potential here to sharpen up forecasting? If you’re talking with a company and you say, “I can sharpen up your forecasts and your estimates and tighten up your variability and your business plan.” How does that translate to value and how do you extrapolate that to here’s the the slack, if you like, that’s built up economically within the system and as a whole that we stand potential to extract out and it’s going back to the misallocation of capital, the inadequate negotiations with suppliers, the margin left on the table because of the numbers aren’t just that reliable.
TN: We just went through this exercise with a manufacturer with about 20 billion dollars of turnover to help them understand. If you look at, say, the nine percent difference that we had in that exercise that I told you. So let’s say we’re working with the manufacturer with a 20 billion dollars and a PE ratio around 20, which is kind of where they’re trading. If instead of a nine percent or even four and a half percent improvement, let’s just say we had a one percent improvement in their materials. That one percent improvement in their costs translates to a three percent improvement in their net income. That’s three percent improvement in their net income translates to a 1.1 billion dollar improvement on their market capitalization.
We’re not going out there saying, “hey, we’re gonna help you save 10 percent of your costs.“ We’re not going out with statements that are that bold. We’re saying, “OK, let’s run a scenario where we help you with a quarter of a percent,“ which would help them add 280 million dollars on to their market capitalization. So procurement management and planning is kind of that tightly calibrated that if we helped this company with 0.25 percent improvement in their costs, keep in mind we’re nine point four percent better than consensus, that actually helps them add 280 million dollars onto their market cap. It’s just exponential.
GC: Well, it’s the leverage effect of earnings per share as you drop those earnings to the bottom line. And so anybody who’s actually measured on EPS or stock price should take a very interested look at this because you’re not selling a hardware, big capital investment, stand up a big department, not stuff. This is about taking the current process, that’s their budgeting, and squeezing out the variability or the error rate and trends that translates directly to value. When you think about it, it’s a complete no-brainer. Like, why would you not do this?
TN: It is. And we’re not going to charge them 280 million dollars to do it. But we could charge for this agile budgeting and forecasting. But we’re not going to.
GC: What you would do is you’d say, we’ll take shares in your company.
TN: I mean, that’s been suggested many, many times.
GC: Yeah, no, I totally get It. I say to oil companies, I’ll sell my services to you based on the price of oil. But the shareholder actually values the volatility on oil pricing. So they’re not prepared to give that away. And I’d be the same. I wouldn’t do that. But on the other hand, the back to this question of untapped potential. The ship, the bulk of the economy is operating off of wildly inaccurate consensus estimates. I think that’s fair to say, I don’t know if that’s accurate or not, but that would be my my conclusion. The bulk is operating off of inaccurate assessments. And so over time, what should happen is we should see a considerable improvement in that, which in turn translates into much better performing economy, allocation of capital and supply chains and so forth.
So you’ve been an entrepreneur now for how long’s it been three years?
TN: It’s five. We started as a consulting firm. It’s been about five years now. We actually started the company in Singapore. I moved it to Texas at the end of 2018. I couldn’t really find the coding talent and the math talent in Asia. I know this sounds really weird, but I couldn’t. And so I relocated the business to Texas in 2018.
GC: Yeah. And the talent pool is rich enough in the United States to fulfill this ambition?
TN: Yeah, yeah. Yeah. Totally fantastic.
GC: And what lessons have you taken away from all of this experience? Would you do it again?
TN: I would do it again. But I would do It differently. Anybody who starts a business has to realize that markets aren’t necessarily ready for radical new thinking. And it really takes a long time to get an idea of this out there. The kind of AI industry and the talk about automation has been around for a long time. But things like this, companies aren’t really ready to just let go of. It takes a lot for them to consider letting go of this stuff.
If your idea is pretty radical, it’s probably to take a while to socialize with an industry. But I would say it’s also, we as a company, we had a staff issue about a year ago, actually, that really shook us. And out of that, we developed our principles and our values. For anybody who wants to do this, you really have to understand what your own principles and values are from early on. It’s not something you wait until you’re 100 people to develop.
That issue a year ago was a very clarifying moment for us as a company. It really forced us to think about what kind of business we wanted to build. And I’m grateful for it, although it was really terrible at the time. I’m grateful for it because we have our values. It’s actually posted on our website. Whenever we recruit new people, that’s one of the first things I send to them and say, “Look, this is who we are. If you’re not comfortable with this, then this is not the right place for you. I’m sure you’re talented, all that kind of stuff. But we really live by this stuff and and those things are important.“
The other thing I would recommend for anybody who’s doing this is you’ve got to play nice with everyone on the way up and you got to play nice with everyone on the way down. It’s easy for tech entrepreneurs to really think a lot of themselves. And I think that’s fun. But it’s also not really helpful in the long run.
There’s a lot that I’ve learned about recruiting leadership teams, finding fit, looking for investors. I have the Asia experience. I have the U.S. experience. The math and the tech around A.I. is almost the easiest issue to solve. With technology, as long as you think big but retain humility, you can do a lot. You have to be bold, but be comfortable with mistakes.
The trick is getting the right team and the right investors who are comfortable with that environment. And if you get the right team and the right investors who are comfortable with that, then it can be much more fun. You actually have a chance at being successful because so many startups just fail. They don’t last a year or two years, much less for five. It’s really, really critical to get the right people.
GC: Yeah, I completely agree. The people and the money, it’s both sides. If the investors don’t have the patience or they’re marching to a different drum like they want short term results, and that’s as much of a death knell for for many startups as a talent talent deficit.
Tony, this has been excellent. Thank you very much for taking the time to join me today on Digital Oil and Gas. And if people want to learn more about Complete Intelligence, where do they go? What’s your website?
TN: Our website is completeintel.com. And we’re on Twitter. We’re on LinkedIn. There’s a lot of information there. And like you did about a month ago, we have a lot of five-minute interviews we do with industry experts and a weekly newsletter. There are a lot of ways to get to learn about us.
GC: Fantastic. Tony, thank you very much. This has been another episode of Digital Oil and Gas. And if you like what you’ve heard, by all means, press the like button and the share button and add a comment, and that helps other people find the show. And meanwhile, tune back in next week, Wednesdays, when we’ll issue another episode of Digital Oil and Gas. This is Canada Day week. So happy Canada Day to my all my Canadian listeners.
And Saturday is Independence Day. It’s July 4th. So, Tony, have a great time on Independence Day. Be socially distant and be safe out there. Thanks again.
