Tempus Resource 2020 Conference Recap: Eric Qin, Qualcomm
Four years ago, Qualcomm Technologies was in search of a unified resource management solution for its entire enterprise. The semiconductor technology company that designs the chips in most cell phones has been the leading technology provider of 2G through 5G. Eric Qin, Director of Engineering, describes Qualcomm’s search, implementation, and the results of choosing Tempus Resource. In a company with over 30,000 engineers, groups organically developed isolated processes that eventually needed to be unified. Teams were using different databases, systems, and tooling—some homegrown and some third-party. This kind of disparate work led to a lack of higher-level project reporting, inability to rollout new projects effectively, and lack of company-level forecasting for roadmaps.
When leaders came together to seek a solution, their success criteria were ambitious. They needed to provide clear insights into non-recurring engineering costs, integrate multiple systems (including HR, finance, product, and projects), and automate project management and engineering operations for peak productivity. Qualcomm wanted enterprise-level, real-time reporting, improved long-term road mapping, and what-if modeling at every level.
The innovative architecture, Eric explains, seamlessly integrated Tempus into their existing systems, then further blended with homegrown Qualcomm tools. One such homegrown tool is a three-tiered application called Tempus Extension Portal used to perform bulk operations, more complex cloning, and other functions that are Qualcomm specific. Additionally, the Tempus database is linked directly to Qualcomm’s data warehouse, an aggregate of resource, product, project, and finance data. Most impressive is the custom reporting function that takes data from all the integrated Qualcomm systems to produce complex and high-value reporting, at any level or enterprise-wide.
Three Year Review
After over three years, it’s clear that Tempus was the right pick for Qualcomm. “This solution streamlined the data flow, improved productivity, and provided insights to help business decisions,” Eric affirms.
Integration was seamless and turned out to be as crucial as anticipated. “No one has time to do manual, repetitive work,” Eric states. Integration and API automation means an end-to-end solution that performs bulk operations and greatly enhances productivity. Additionally, Tempus provides for consistent company-level rollup with the right amount of flexibility to give every business unit custom fields and workflows to meet their unique needs.
We appreciate Eric Qin for leading us through Qualcomm’s journey to an integrated, enterprise-level resource management solution. We’re sure many others hoping to see the same impressive gains at their own organizations will benefit from Eric’s wisdom and experience.
Transcript: A Unified Resource Management Solution
Thank you Greg and Shawn and for this opportunity and for hosting this Resource Management Conference. A lot of speakers have given great presentations and a lot of ideas are very interesting, so let me share my part. Just a quick introduction, my name is Eric Qin and I work for Qualcomm. We are based in San Diego, California. I am a director of engineering and my role is I am responsible for the program and management related to tools and the process here at Qualcomm and that’s why I’m involved in resource management.
For today I have a few topics here. We’ll go through a quick introduction. We’ll talk about what is the solution we put in there and what does the architecture look like. Then we’ll look at our business analytics, the reporting solution, which is how the business users get value out of this resource management solution. I will talk about our adoption journey, how did this solution grow over the last four years and we’ll do a summary about what we did well and then what we are planning to do next. I hope some of this information will be useful to you. If you are a new Tempus user or maybe you are a seasoned Tempus user for many years like us at a large enterprise, hopefully there’s something useful.
You see us everywhere you see 5G
Okay, about Qualcomm. What do we do? Qualcomm is mostly a semiconductor technology company. The number one thing we do is we design the cell phone chips so if you ever use a mobile phone, you are probably using our chips and using our software solutions as well as using our communication technologies. We have been a company for 35 years and we have been the leading technology provider from the 2G to 3G to 4G. Now the worldwide is transitioning to 5G and we are the leading provider for 5G also. Whenever you pick up your phone, you use your Wi-Fi, whether you’re working from home or you’re doing your Zoom meetings, or hospitals using telemetry healthcare, you are using our technologies.
How did this start? Qualcomm being a large R&D company mostly run by engineers and it’s very organic. The way I look at this is it’s almost like a gigantic startup with 30,000 engineers. It’s a very organic culture so what happened was along the way every group, based on your need, we develop your own process and we develop your own tooling so about four years ago a bunch of us gathered together because we had some issues to say hey, what we need to do is a higher level project reporting. We don’t really have a good way of doing it because all of these teams are using their own databases and they’re using their own tools. Some of these tools are homegrown, some of these are third party. We don’t really have a good way to roll out the NRE for certain projects. Sometimes one of the projects can be using a few thousand engineers for multiple years, so we don’t have a way to do that roll-out. When we try to forecast our roadmap at a company level we also have troubles. Also the hardware guys and the software guys are using different systems so there were all kinds of issues.
Then we got together and we said okay we need to figure out a unified resource management solution for the entire Qualcomm. So we started by looking at different solutions whether it’s our home group or whether it’s a third party, so what are the criteria we want. One thing was this new solution needed to provide a clear NRE insight for project in a timely manner at both the time we review and the plan for a project as well as, some of the projects last for several years so along the way you will have new customers, you will have new features added. So when the project is executing we also need to compare what is our project cost versus what we approved for at the beginning so we can adjust accordingly.
Second thing is we know from the beginning we have to build an integrated solution. It cannot be a resource management tool on its own island. It has to be connected with all the other Qualcomm systems. The HR system has all the resources, finance projects have all the finance side of things, products (that’s where all our new products are) and with the product milestones, project size (that’s where we do project management). So you break the project into schedules and assign the tasks to different people, etc. And on top of this we know very importantly we need to put a BI layer. This is where the business intelligence comes in because with all this data you’ll have to translate into a way people can easily consume and then make a decision out of it. That’s where we say, hey it has to be an integrated solution for efficiency.
Automation. Most of the Qualcomm people are engineers, whether you are a hardware engineer or a software engineer and nobody likes to do repetitive data entry work. So that’s why we said it has to be automated as much as possible including bulk operation basically because nobody likes data entry job.
The next one is it has to be able to support enterprise level real-time report. So we always at any time whether you are an individual tech team with let’s say 50 people or you’re at a BU level with a few thousand people or you are at the entire Qualcomm level with 30,000 people, we need to be able to say for a project, for a country, for a BU what is my capacity, people capacity. What is my current consumption or are people going to allocate you to the project and how does this track against my time sheet actuals? We use money chart to allow people to compare this anytime you want and filter down to your group, your project, your country so that has been very useful.
One of the automated goals, and we are not totally there yet so we’re still working on it, is to improve the long-term roadmap planning at the company level, the BU level, and the technology team. This is where we really want to leverage the what-if modeling and that will allow us to plan out our project and our resource needs for two, three, five years down the road and that will be very, very useful.
So this is where I will walk through the solution architecture. So let’s say you have a bunch of Tempus users. The user typically will use a Web interface. This Web page can be the Tempus native page where you can tell is the BPA flat grade or can be using our own third-party Web page so we developed a new thing called the Tempus extension portal. It is a three-tiered application where people can use the three-tiered extension portal to do bulk operations that are Qualcomm specific. So a user coming here using these Web pages to do resource management or what-if modeling so here you can see in this box here we have the Tempus database itself, so this is a native solution. We developed our own extension on top of it called the Tempus extension portal. It has its own database, it has its own services, it also has its own Web portal where people can perform more of the complex cloning, some of the bulk changes, etc. It’s all automated to Qualcomm work.
So, in order for people to use this solution efficiently we said it has to be integrated which means we have to pull in all the other data into Tempus and make it available for people to see when they do their forecasting. So what are the things we need to put in? First all the people, the resources, so we get all this from the HR system. New people joining Qualcomm every day, people leaving, people changing departments, our integration will take care of those changes every day. This means any time you go into Tempus you see the latest set of resources in their refresh.
Products. So what are the new products we are developing and what are the key milestones because when people forecast the resources to a product, they need to know the milestone. In order to meet this milestone, I need to forecast so many people with this skill set to do that work and get it done before that milestone. So we need to see that also.
Projects. So these are the projects with the schedules and some additional information we will get some in here also and then the finance side of things. Finance decided you have the time sheets, you have the finance charge code, so that’s where you’re pulling also. We have a data warehouse in Qualcomm. Actually there are many warehouses but I’m just showing one here. Basically all these systems, rather than doing point-to-point integration, we push them all into a data warehouse and then we push this data into Tempus. So when you log in as a resource manager you will see all your resources, all your products, and all your finance information. Then you can just do your forecasting and all of these changes flowing continuously so you don’t have to worry about manual maintenance.
So now as a resource manager I’m in Tempus. I can see all my people capacity by setting filters and then I can enter my project demand and I can allocate my people to projects. Once I do that, that data is pushed back into the data warehouse. Why do we do that? Because this is reporting solution. Don’t get me wrong. We do use the Tempus native reporting solutions like the pupid grid and also the barcode chart. We use that a lot, but for certain more complex reports, we have to do it outside. That’s where we go with Tableau. There are two main reasons why we go with Tableau. One thing is it is a dedicated BI tool, it can handle a lot more data, and it also has the capability to allow you to define very complex reports as well as the data you want behind the scenes. So this means we can generate a lot of complex visualizations as well as we can compare time phase to data. The second reason we use Tableau is in order to generate this report, we not only need the Tempus data, we also need a lot of the data from other systems from here. Also there’s also other systems I’m not even listing here. In order for this report to provide a better value, we have to combine all this data together so that’s why this cannot be sitting in data timeline. This has to be sitting in a data warehouse or a data lake.
Analytics – Tableau
So, I’ll give you some examples of the report we have. I think at this point we have about 50 Tableau reports overall but the majority of the reports fall into three categories. The first one is table so people like Excel, people like tables because it’s useful and it’s also useful for very detailed data analysis. So here we allow people to come in here and apply their filters, zoom into the project or the resources or the department secure and then you can deep dive into this or you can basically do whatever you want and if you really want, you can download this to Excel and do whatever you want. The key here of this table is we allow the user to define on their own what are the rows and what are the columns. We give them that customization they can control, so it’s almost like you can define your own pivot table for your own needs so that’s what people really like.
This one in the center is called the mountain chart. This is the most valuable and the most useful report we have ever done in our resource management solution. So what this does is it allows you to compare your capacity versus your allocation versus your actuals. Here once you apply your filters, you have your capacity line and then each of these mountain layers so by default it’s a project, so you can say I have 3,000 engineers and then how am I spending them? I’m spending on project one, project two, all the way stacking up, or you can say I don’t want to show them by individual project. I want to show them by country. I want to show them by department. Or you can even say I want to show them by BU. Or you can say I want to group the projects by portfolio so people can group the layer controls. So that’s another very useful thing people have.
The third kind of report we do is comparing time phase to data. So, for example you have a project. Originally when we approved the project it has a base budget of let’s say it’s this lower line in here. How we are going to spend the number of dollars or how we are going to spend in the man-months across this curve? Each of these will be a month. And later, let’s say six months into the project, we got some new customer interest, we are adding more features so the project scope changed and we need to do a re-plan. When we do a re-plan we changed the budget and we reviewed that, we agreed on that so we have a new plan of record budget so we want to show that. Now three months have passed again so what is the current forecast on that project? How does that forecast compare against the base budget and the latest approved budget month by month? This chart gives you exactly that. Why is this useful? Because this gives us the capability to ask are we always under-budgeting for a certain type of project? If yes, what is the reason? Maybe because when we define the project we always miss certain customers, we miss certain features, or is it because this kind of project involves a certain type of complexity that we don’t understand at that time? If that’s the case, the next time we plan for a similar project we need to increase the forecasting in those areas or maybe we should put in some buffers so that will allow us to do more accurate planning.
So we have a solution in there. The initial solution rolled out in 2017 and we have been constantly working on developing and enhancing the solution and that work is still ongoing. So this is how our user trend happened over the last three and a half years. We officially rolled out Tempus at Qualcomm here on April 1, 2017. When we rolled it out people asked if it was a joke because we sent out as a notification on that day. Here’s a new tool. Here’s how you should do it. They said this is a joke. No it’s not a joke because I intentionally chose that day just to get people’s attention. When we started it, this dark blue line is the number of resources we started with and this is how many thousand engineers. So we started with managing about 6,000 engineers and then we grew here and then we onboarded the software teams in 2019 and then at this point we have about 28,000 engineers managed in Hong Kong (i.e., Tempus here) and this light blue line is the number of projects. We started at about 600 projects. At this point we have about 2700 projects where we are forecasting people to. Some of these projects can be really big; some of these can be really small. This line is the planners, so this means how many resource managers are doing forecasting in the tool. This does not include any viewers who are the read-only people because for all those viewers we kick them to the Tableau side because they are not changing data so we just send the straight to the Tableau. All the reporting is over there. So we started with about 200-ish planners and at this point we have about 1100 active planners using the tool, changing forecasts.
Reports. So this is a Tableau solution I am talking about here. When we initially rolled out in 2017 we did not have Tableau solution at that time. I think we built our first Tableau solution in 2018 and then it keeps growing and growing, so at this point every month we have about 11,000 Tableau user sessions so that’s how you can see it’s really grown a lot.
As I mentioned earlier it’s all about organic growth because at the program, as I mentioned already, it’s like a gigantic start-up. We don’t want to reduce people’s creativity so we have to give the engineers freedom so that’s why we encourage organic growth. We loaded the solution to some of the teams and then other groups hear about it. They see it. They can tell the value and then we start onboarding another set and it just keeps going on like this.
Global Users. Qualcomm being a global company, our planners and our users are also global. So a lot of people in the United States for sure, a lot of people in India, in Europe, in China, pretty much all the major countries you can think of we have users from those. That also means our support team needs to be global because it’s basically a 24/7 thing.
Biz and Engineering. Most of these planners, they are user engineering leads or they are the program managers so that’s the primary user set. For the high level executives, we send them to Tableau because they are looking for a prettier chart and something you can easily consume so that’s when we send them
So now after three and a half years, when I look back I know what went well and what are the things we should do next. The number one thing is clearly that Tempus was the right choice as the foundational tool because Tempus natively is very good, and also with the Tempus API a lot of things can be done to automate the process to integrate with other systems
Integrated solution. If there’s only one thing you remember, please remember that it needs to be an integrated solution. That is really the key because at least in our situation, because all the users are PMs or engineering leads, nobody has time or nobody enjoys manual repetitive work. So by integrating all the different systems together we provide an end-to-end solution where we pull in all the necessary data for you into Tempus. And then within Tempus we use the API to help you to do the bulk operations. And then we provide to you the overall reporting solution where you can look at the chart, make sense out of it and then make your business decisions based on that chart. So that’s what this integrated overall solution provides value and when people see value they say this makes sense. Okay, I don’t consider my data into work to be wasted because at the end I get a value out of it. I can make decisions based on these reports. So that’s why integration is a key.
Company level roll-up consistency, but you still need to allow the business unit or organization to have their own flexibility. So this is where we say that there are certain ground rules at the company level you have to follow. Those we cannot break because if we break, we break the entire company level reporting on roll-out but for each of the organizations or business units, they have their own needs so we give them an additional custom field, we give them additional work flows so they can meet their own organization or specific need.
Advanced reporting. I think I already talked about enough on the Tableau. I think that was our key success factor in this overall solution.
Automation with API for productivity. That’s the way you give people productivity. Everywhere we can automate using the API, we automate rather than doing manual work.
Global support team. We do have a global support team here and in India so during our nighttime if our users in Europe or in China or India have questions our India team will help them.
Continue tooling enhancements. So what’s next? Where do we want to go from here? One thing of course is to continue the tooling enhancements. We have been on this journey for three and a half years now and I think with the constant user adoption and always with more and more enhancement people want to do, the development will be continuing I think for the next two years.
What-if modeling. So this is one of the areas we really want to invest in in the next year or so because I can see a lot of value out of this, but what-if modeling can be very complex especially when you try to do this at a large enterprise with very complex projects and all the people and the product dependencies. So that’s where we still have a lot of work that needs to be done.
Business through automation. That is a given.
Data quality. That is another thing. The more data you have in the system the more issues you’re going to have. There’s inaccurate data, so we’re going to spend a lot of time to put a road in there to validate the data and also try to prevent the invalid data getting into the system. If your data is not high quality, then the reporting will be showing the wrong information. Then the overall value of the solution will be diminished.
Use ML to predict and alert, useful assistant. The last part, this is something I’ve been trying to do for quite a while, but our developers were always on building new features so we didn’t really spend too much time on this, but the one thing we always want to do is, rather than relying on the program managers to alert that this project might have an issue with budget overrun or future overrun or certain schedule risk, we would like to use machine learning to be able to predict those things even before the program managers can see that. So the reason why I want to do this is because within my team in addition to Tempus I have a lot of other systems and I have requirement data and all the product data so by combining these I feel we can provide more value to our business.