QUICK FILTERS
Featured
8 min read
Gina Calvert
Are you as intentional about measuring the value of your data infrastructure and models as you are about building them? In the video below, our Solutions Architect Phil Schrader recently revealed at People Analytics Summit in Toronto the importance of (and some strategies for) using analytics to evaluate the impact of your analytics investments. From leveraging machine learning to track improvements to thinking creatively about integrating predictive models into everyday workflows, you'll gain insights on how to apply analytics to your own analytics. Short on time? We’ve summarized his presentation for you below. The Core Problem: Evaluating Data Investments When we talk about people analytics, we often focus on the tools, processes, and models that drive better decisions. But what happens when we turn that lens inward—when we use analytics to assess the very work of analytics itself? The idea is simple: if we’re investing in building data infrastructure and models, we should be just as intentional about measuring the value of those investments. Anyone leading a people analytics team knows the balancing act. On one side, there’s the pressure to deliver quick insights, the kind that keeps operations running smoothly. On the other side, there’s the longer-term need to build out robust data systems that support advanced analytics. Yet, as essential as these data initiatives are, we often struggle to quantify their value. How do we measure the ROI of building a data lake? How do we ensure that the data we’re collecting today will pay off down the road? Solution: Analytics About Analytics Here’s where we can take a different approach—by applying analytics to our own analytics. The falling cost of technical work in machine learning (ML) has opened up new possibilities, allowing us to embed these tools within our day-to-day operations. Instead of just using ML models for predictions, we can use them as a means to measure how good our data is and how effective our processes are. Essentially, we can start to think analytically about how we do analytics, especially when it comes to creating a predictive model that measures improvements over time. A Concrete Metric: Precision, Recall, and the F1 Score The foundation of this approach lies in the well-known metrics used to evaluate machine learning models: precision, recall, and the F1 score. In brief: Precision asks: When the model makes a prediction, how often is it correct? Recall asks: Out of all the events that should have been predicted, how many did the model actually identify? The F1 score strikes a balance between these two metrics, offering a single number that reflects how well your model performs overall. By tracking this metric, we can gauge the quality of our data and see how incremental improvements—like adding new data sources—translate into better predictive power. This kind of measurement becomes crucial as we think about the future of machine learning and how it integrates into everyday operations. Building Analytics for Growth This method doesn’t just give us a way to measure progress; it gives us a framework to demonstrate that progress in tangible terms. Start with the basics—core HR data like job titles, tenure, and compensation. As you layer in additional data points—learning metrics, performance reviews, engagement scores—you can observe how each new addition boosts your model’s F1 score. It’s a practical way to quantify the value of your data and justify continued investment. The Changing Landscape: Embedding Predictive Models Predictive modeling no longer needs to be a separate, resource-intensive project. As the tools become more accessible, we can embed this capability directly into our workflows. Think of it as using predictive models the way we use pivot tables—regularly, as a quick check to see how well our data is performing. This kind of embedded analytics allows us to experiment, iterate, and find creative ways to leverage machine learning without overcommitting resources. With AI continually reshaping business practices, this shift will allow teams to use predictive models in increasingly versatile ways, driving more efficient decision-making. Beyond Traditional Metrics: Rethinking the Value of Data By adopting this approach, we’re able to ask—and answer—a critical question: How valuable is our data, really? If we can demonstrate that our data is increasingly effective at predicting key outcomes like employee turnover or high performance, we’re no longer just talking about data quality in abstract terms. We’re providing a concrete metric that resonates with stakeholders and gives us a way to collaborate more effectively, whether it’s across HR functions or with external vendors whose data feeds into our models. Looking Ahead: Embracing Innovation as Costs Fall The future of AI and the workplace is advancing quickly, blurring the line between strategic and routine applications. What was once a complex, time-consuming effort will soon be something we do without a second thought. This shift requires a mindset change—being open to ideas that may seem wasteful or unconventional today but could become standard practice tomorrow. The key is to embrace this shift and look for new, innovative ways to use predictive analytics. In summary, by taking an “analytics for analytics” approach, we gain more than just better models—we gain clarity on the value of our data investments. The ability to measure progress in predictive power isn’t just a technical exercise; it’s a strategic advantage that drives smarter decision-making across the board. Not sure where to start? Download Key Questions to Ask When Selecting an AI-Powered HR Tool to get the answers you need. Download Your Buying Guide Now
Read Article
Featured
4 min read
Richard Rosenow
The buzz around Artificial Intelligence (AI) in the workplace is growing louder by the day. As organizations worldwide attempt to harness this revolutionary technology, particularly in the realm of Human Resources (HR), a fundamental question arises: Is our workforce data truly ready for AI and Machine Learning (AI/ML)? The Reality of Data Readiness for AI and ML In our modern business environment, HR teams are making use of workforce data for a variety of purposes. Traditionally, these teams had focused on extracting data for reporting in the form of monthly extracts or daily snapshots. This approach, while useful for traditional needs, falls short of the data needs for AI and ML. That’s because data preparation for AI isn’t just about collecting and storing data to review later; it's about curating data in the right way to effectively train sophisticated models. AI tools today are highly complex and capable of predicting patterns with remarkable accuracy. However, vast amounts of high-quality, curated data are required to effectively train those models. The quality and relevance of the data are critical for the fine-tuning needed for specific tasks or domains like our use cases in HR. The Need for a Paradigm Shift From this perspective, most HR datasets and HR data stores that we had previously prepared are not ready for AI and ML (whether it's generative AI or "traditional" predictive AI). Without appropriately prepared training data, the algorithms we hope to launch will fall short in their learning. Potential benefits of AI in HR—from recruitment optimization to workforce alignment with business goals—could remain untapped or, worse, lead to unintended consequences if models are trained on poor or incorrect data. Preparing your HR team for this new phase of work isn’t just about adopting new technologies; it's a paradigm shift in how we think about and handle data. This is even more pivotal in the areas of MLOps and LLM operations when we try to deploy these models at scale in a repeatable fashion. We’re going to start to hear more about these terms and the operational needs of machine learning in the near term future and it’s HR’s responsibility to stay on top of the nuances in this space. The First Step: Preparing and Unlocking Your Data Data extraction is one of the most essential parts of preparing for AI and ML. We address the foundational importance of this step, robust data preparation and management, in our blogpost 5 Tips for Getting Data Extraction Right. It explores in greater detail these 5 action steps: Prioritize and align extracted data with the needs of the business Be thoughtful about what you extract Build the business case to pull more Automate your extractions Extract for data science, not just reporting The paradigm shift and these tips can help HR teams more effectively and efficiently adopt AI practices that will drive business value and insights. Why One Model Stands Out in People Analytics AI The final key in preparing for AI and ML is having the right technology in place to build a fine-tuned model that meets your company’s unique needs. One of the main reasons I joined the One Model team stems from their foresight and commitment in this area. Due to that investment, we're now the only people analytics vendor with a machine learning platform that runs on a data model tailored to your firm, not just last-minute AI features. This distinction is vital. And "One Model" isn’t merely about preparing data for AI models; it’s an end-to-end platform encompassing data management, storytelling, model creation, evaluation, deployment, and crucially, audit-ready and transparent tools. Our platform empowers HR teams to manage and deploy customized ML models and MLOps effectively, beyond the traditional scope of data engineering teams. The dialogue around AI, ML, and MLOps in HR is already in full swing. Staying informed and engaged in this conversation is crucial. If you wish to delve deeper or discuss strategies and insights in this space, I, along with the One Model team, am more than willing to engage. We're keen to hear how your team is navigating the intricate landscape of MLOps in HR. Essential Questions to Ask When Selecting an AI-Powered HR Tool Learn the right questions to ask to make the right decisions as you explore incorporating AI in HR.
Read Article
Featured
7 min read
Steve Hall
Reporting specialists and data analysts are often required to predict the future for stakeholder groups. They do this through a variety of models, including forecasting and annualization. Although both methodologies aim to predict future values, their applications and the mathematical logic behind them vary significantly, catering to different business needs. What is Annualization and Its Significance? Annualization is a mainstay for finance and accounting but there are situations where it may be useful in HR contexts. It can be used to estimate year-end values for turnover rates, total new hires, and job openings filled, based on current data. Annualization works well when: There is little volatility in the metric across time periods There is little seasonality in the metric The metric is not likely to trend upward or downward during the course of the year Simplify Annualization with One Model One Model streamlines the computation of annualized metrics. By selecting the "Year to Date" option and "Annualize" in "Time Functions," the system will only consider the current year's data, offering a clear example of annualization at work. The Case for Forecasting Forecasting provides several benefits over annualization. While annualization typically only considers data points from the current year, forecasting can: Utilize data from a much wider time frame and range of data points Factor in seasonal fluctuations and trends Provide a more nuanced view of potential future states with confidence intervals, which is especially valuable for HR metrics that exhibit variability (e.g., number of hires, number of terminations, and termination rates). Simplify Forecasting With One Model One Model simplifies forecasting with its Embedded Insights feature. Just create a time-series line graph for your metric and use the feature to extend your forecast to the year's end. Increasing the number of data points, by adjusting the time metric from monthly to weekly or daily, for instance, can enhance forecast accuracy by capturing shorter-term cycles that may be present in the data. Including data from at least 30 data points will improve the accuracy of your forecasts and if annual seasonality is present, including data covering two or more years will also improve accuracy. You can adjust forecast parameters to align the final forecast period with the year-end. After running the forecast, simply click on the last data point in the visualization to see the forecasted value and its confidence interval. For more complex situations where the current year data pattern is expected to shift relative to last year’s pattern, One AI can be used to create a predictive model that incorporates additional internal and external features to improve accuracy. Making the Choice: Annualizing or Forecasting? Annualization and forecasting each have their strengths and weaknesses. Deciding between them depends on your data and your stakeholders’ needs. Sometimes a rough approximation is good enough; other times, a precise estimate or a range of values (e.g., a confidence interval) will be required. Annualization Forecasting Only considers data from the current year Can leverage data from multiple years Only needs a single month of data to start the estimation process One Model will need at least 4 data points to produce a forecast, but forecast accuracy suffers with so few data points unless the metric progresses in a very linear fashion Does not adjust for seasonality or trend Accounts for trends and seasonality Very simple approach requiring little input regarding computations and easy to understand More sophisticated approach that may prompt questions from end-users (luckily One Model provide embedded information describing the forecast) Estimates made early in the year are likely to be inaccurate Estimates made early in the year are likely more accurate than Annualization, especially when data from the prior year are utilized Will always underestimate or overestimate if a trend is present Can produce more accurate results even when trend is present Alternatives and Strategic Adjustments Alternatives like the 12-month rolling average provide another strategy for estimating year-end values, accommodating changes anticipated over the year. For specific metrics, like annual turnover, manually adjust the year-end prediction using expert analysis or the expected effects of internal actions. Depending on the metric being forecasted, it may also be reasonable to manually adjust the year-end value based on general projections for the year. For instance, to predict next year's annual turnover, start with the current end-of-year rate and refine it using projections from external experts or by considering the expected effects of internal measures aimed at reducing turnover. One Model Simplifies Forecasting and Annualization You might encounter scenarios where estimating year-end values for a metric is necessary. Although predicting these values with absolute precision is challenging, One Model can generate reasonable estimates, bearing in mind that sudden changes mid-year could significantly affect forecast accuracy. In practice, forecasting, particularly with One Model's Embedded Insights, tends to be more effective than annualization, especially at the start of the year. However, the accuracy of forecasting is impacted by decisions related to data inclusion and model parameters. Forecasting may also require a bit more effort to maintain, albeit minimal. Fortunately, One Model simplifies the use of both annualization and forecasting. In fact, using both methods to create estimates can be practical. When the results are close, opting for the annualized figure might be preferable for its simplicity. If results differ, the underlying data should be evaluated and the method that best aligns with the data’s characteristics should be used. One Model has you covered regardless of the situation you face and the approach you prefer or choose.
Read Article
Featured
5 min read
Joe Grohovsky
In a recent editorial (here), Emerging Intelligence Columnist John Sumser explains how pending EU Artificial Intelligence (AI) regulations will impact its global use. A summary of those regulations can be found here. You and your organization should take an interest in these developments and yes there are HR legal concerns over AI. The moral and ethical concerns associated with the application of AI are something we must all understand in the coming years. Ignorance of AI capabilities and ramifications can no longer be an excuse. Sumser explains how this new legislation will add obligations and restrictions beyond existing GDPR requirements and that there is legislation applicable to human resource machine learning. The expectation is that legal oversight will arise that may expose liability to People Analytic users and their vendors. These regulations may bode poorly for People Analytics providers. It is worth your while to review what is being drafted related to machine learning and the law as well as how your current vendor addresses the three primary topics from these regulations: Fairness – This can address both training data used in your predictive model as well as the model itself. Potential bias toward things like gender or race may be obvious, but hidden bias often exists. Your vendor should identify biased data and allow you to either remove it or debias it. Transparency – All activity related to your predictive runs should be identifiable and auditable. This includes selection and disclosure of data, the strength of the models developed, and configurations used for data augmentation. Individual control over their own data – This relationship ultimately exists between the worker and their employer. Sumser’s article expertly summarizes a set of minimum expectations your employees deserve. When it comes to HR law, our opinion is that vendors should have already self-adopted these types of standards, and we are delighted this issue is being raised. What are the differences between regulations and standards? Become a more informed HR Leader by watching our Masterclass Series: Why One Model is Preferred when it comes to Machine Learning and the Law? At One Model we are consistently examining the ethical issues that are associated with AI. One Model already meets and exceeds the Fairness and Transparency recommendations; not begrudgingly but happily because it is the right thing to do. Where most competitors put your data into a proverbial AI black box, One Model opens its platform and allows full transparency and even modification of the AI algorithm your company uses. One Model has long understood the HR law and how the industry has an obligation to develop rigor and understanding around Data Science and Machine Learning. The obvious need for regulation and a legal standard for ethics has risen with the amount of snake oil and obscurity being heavily marketed by some HR People Analytics vendors. One Model’s ongoing plan to empower your HR AI initiatives includes: Radical transparency. Full traceability and automated version control (data + model). Transparent local and model level justifications for the predictions that our Machine Learning component called OneAI makes. By providing justifications and explanations for our decision-making process One Model builds paths for user-education and auditability for both simple and complex statistics. Our objective has been to advance the HR landscape by up-skilling analysts within their day-to-day job while still providing the latest cutting edge in statistics and machine learning. Providing clear and educational paths to statistics is in the forefront of our product design and roadmaps, and One Model is just getting started. You should promptly schedule a review of the AI practices being conducted with your employee data. Ignoring what AI can offer risks putting your organization at a competitive disadvantage. Incorrectly deploying AI practices may expose you to legal risk, employee distrust, compromised ethics, and incorrect observations. One Model is glad to share our expertise around People Analytics AI with you and your team. High level information on our OneAI capability can be found in the following brief video and documents: https://bit.ly/OneModelPredictiveModeling https://bit.ly/OneModel-AI https://bit.ly/HR_MachineLearning For a more detailed discussion please schedule a convenient time for a personal discussion. http://bit.ly/OneModelMeeting
Read Article
Featured
8 min read
Dennis Behrman
Ever play with a Magic 8 Ball? Back in the day, you could ask it any question and get an answer in just a few seconds. And if you didn't like its response, you could just shake it again for a new prediction. So simple, so satisfying. Today's HR teams and businesses obviously need more reliable ways of predicting outcomes and forecasting results than a Magic 8 Ball. But while forecasting and predicting sound similar, they're actually two different problem-solving techniques. Below, we'll go over both and explain what they're best suited for. What is HR Forecasting? At first glance, the Magic 8 ball "predicts" or "forecasts" an answer to your question. This is not how forecasting works (at least, for successful companies or HR departments). Instead, HR forecasting is a process of predicting or estimating future events based on past and present data and— most commonly—by analysis of trends. "Guessing" doesn't cut it. For example, we could use predictive forecasting to discover how many customer calls Phil, our product evangelist, is likely to receive in the next day. Or how many product demos he'll lead over the next week. The data from previous years is already available in our CRM, and it can help us accurately predict and anticipate future sales and marketing events where Phil may be needed. A forecast, unlike a prediction, must have logic to it. It must be defendable. This logic is what differentiates it from the Magic 8 ball's lucky guess. After all, even a broken watch is right two times a day. What is Predictive Analytics? Predictive analytics is the practice of extracting information from existing data sets in order to determine patterns and trends that could potentially predict future outcomes. It doesn't tell you what will happen in the future, but rather, what might happen. For example, predictive analytics could help identify customers who are likely to purchase our new One AI software over the next 90 days. To do so, we could indicate a desired outcome (a purchase of our People Analytics software solution) and work backwards to identify traits in customer data that have previously indicated they are ready to make a purchase soon. (For example, they might have the decision-making authority on their People Analytics team, have an established budget for the project, completed a demo, and found Phil likeable and helpful.) Predictive modeling and analytics would run the data and establish which of these factors actually contributed to the sale. Maybe we'd find out Phil's likability didn't matter because the software was so helpful that customers found value in it anyway. Either way, predictive analytics and predictive modeling would review the data and help us figure that out — a far cry from our Magic 8 ball. Which Do You Need? Which do you need—forecasting or predictive analytics? The short answer: probably both. Forecasting helps you project trends—like how headcount or attrition rates might shift over time. It’s great for setting expectations and planning ahead. Predictive analytics, on the other hand, digs into the "why" behind those trends. It identifies patterns and flags specific risks or opportunities—like which employees are most likely to leave or what factors drive engagement. Together, they give you both the big picture and the specific actions to take. Ready to explore predictive analytics? We can help you figure it out and get started. Connect with us. Forecasting vs. Predictive Analysis: Other Relevant Terms Machine Learning - machine learning is a branch of artificial intelligence (ai) where computers learn to act and adapt to new data without being programmed to do so. The computer is able to act independently of human interaction. Read Machine Learning Blog. Data Science - data science is the study of big data that seeks to extract meaningful knowledge and insights from large amounts of complex data in various forms. Data Mining - data mining is the process of discovering patterns in large data sets. Big Data - big data is another term for a data set that's too large or complex for traditional data-processing software. Learn about our data warehouse. Predictive Modeling - Predictive modeling is a form of artificial intelligence that uses data mining and probability to forecast or estimate more granular, specific outcomes. Learn more about predictive analytics. Descriptive Analytics - Descriptive analytics is a type of post-mortem analysis in that it looks at past performance. It evaluates that performance by mining historical data to look for the reasons behind previous successes and failures. Prescriptive Analytics - prescriptive analytics is an area of business analytics dedicated to finding the potential best course of action for a given situation. Data Analytics - plain and simple, data analytics is the science of inspecting, cleansing, transforming, and modeling data in order to draw insights from raw information sources. People Analytics - All these elements are important for people analytics. Need basics? Learn more about People Analytics. About One Model One Model’s People Analytics solutions help thriving companies make consistently great talent decisions at all levels of the organization. Large and rapidly-growing companies rely on our People Analytics platform and One AI because they takes all of the heavy lifting out of data extraction, cleansing, modeling, analytics, and reporting of enterprise workforce data.
Read Article
Featured
8 min read
Josh Lemoine
With the introduction of One AI Recipes, One Model has created an intuitive interface for no-code machine learning in people analytics. One AI Recipes (Recipes) are a question-and-answer approach to framing your people data for predictive models that answer specific people analytics questions. Adding this capability to the existing One AI automated machine learning (autoML) platform results in a more accessible end-to-end no-code solution for delivering AI from right within your people analytics platform. We call them Recipes because they walk you through each of the steps necessary to create a delicious dish; a predictive model. Simply select the ingredients from your data pantry in One Model then follow the steps in the Recipe to be guided through the process of creating a successful model. Recipes democratize the production and reproduction of AI models with consistency, accuracy, and speed. Understanding some of the terminology used above and how it relates to One AI will be useful in explaining why Recipes are so useful. What is a no-code machine learning platform? “No-code machine learning platform” is somewhat of a vague term. The definition is pretty straightforward. A no-code machine learning platform is a tool that enables you to apply artificial intelligence without writing any code. It provides a guided user experience that takes business context as an input and produces predictions and/or causal inferences as output. Where it becomes vague is in the range of complexity and flexibility of these platforms. On one end of the spectrum, there are simple-to-use AI builders where the user answers a few questions and is presented with predictions. These tend to only be useful in very standardized use cases. There is often very little transparency into what the machine learning model is actually doing. On the other end are the complex and powerful platforms like Azure ML. Azure doesn’t require writing code and is also very powerful and flexible, but it is also complex. Anyone without a working knowledge of data science would be hard-pressed to create trustworthy models on platforms like this. One AI is aiming at the sweet (no dessert Recipe pun intended) spot on the spectrum. Being designed specifically for people analytics, it allows us to leverage the question-and-answer approach of Recipes. Experienced Chefs can still toss the Recipe aside and cook from scratch though. The One AI kitchen is well stocked with machine learning tools and appliances at its disposal. What is autoML? AutoML is a series of processes or “pipeline” that performs data cleaning and preparation, algorithm selection, and parameter optimization for machine learning. Performing these tasks manually can be labor intensive and time-consuming and requires expertise in data science techniques. AutoML automates these tasks while still delivering the benefits of machine learning. One AI has always provided an autoML pipeline, albeit one where any default setting can be overridden. Even so, there were two areas where we knew we could improve: 1. The data structure for analytic purposes is not the same as the data structure necessary for machine learning. Performing machine learning on data in One Model at times required additional data modeling, a task performed by an expert. 2. Framing up the problem and interpreting the results often required an expert to be involved to ensure accuracy and coherent insights. Recipes address these challenges. Recipes both re-frame the data in a way that a machine learning model can work with and provide a coherent statement that explains both what the model will be predicting and how it will be doing so. How can you benefit from One AI with Recipes? Resource Savings Recipes lighten the load on the technical resources that are likely in high demand at your organization. People analytics is a key strategic business function, yet most people analytics teams aren’t lucky enough to employ Data Engineers, Data Scientists, and Machine Learning Engineers. These teams often fight for the same technical resources as other teams for people who are very talented but can’t possibly possess a deep understanding of all of the different areas of business in the company. Predicting and planning for outcomes has become a key deliverable of people analytics teams, yet they’re often not well equipped to succeed. Companies are increasingly looking for software for automation in HR. Machine learning tools are making great strides in taking business context as an input and producing useful insights as an output. The full realization of this functionality is no-code machine learning platforms. Time Savings With Recipes, time-to-value for machine learning from your people data is substantially reduced. The difference in time required to manually perform this work versus leveraging a no-code machine learning platform is stark. It’s weeks to months vs. hours to days. Even if you have Data Scientists on staff that have the skills necessary to build custom predictive models, they can save time by prototyping in a no-code environment. Interpretability Having the clear statements that Recipes provide that explain what it is you’re predicting and how you’re going about it makes the results easier to interpret. Contrast this with manual machine learning where details can get lost in translation. This prediction statement is in addition to the exploratory data analysis (EDA), model explanation/performance, and Storyboards that One AI provides. One Model also employs a team of experts in the ML and AI space that are available to assist if uncertainty is encountered. Transparency Since One AI is part of One Model, your model configuration and performance data is available in the same place as the predictive or causal data and your people data (at large). Also, your models are trained on YOUR data. These are not “black box” models. At One Model we emphasize making model performance data easily available anywhere predictive data or causal data is included. Compliance As a One Model customer, your potentially sensitive employee data resides in the same place as your machine learning. You do not need to export this data and move it around. On the flip side, the output from your models can be leveraged in your Storyboards in One Model without exporting or moving sensitive data outside of your people analytics solution. The predictive outputs can even be joined to your employee dimensions to help you identify where risk sits. Control and Flexibility Users have the option of configuring data and settings manually in a very granular way. Just because One AI offers a no-code option for creating machine learning models doesn’t mean you’re tied to it. Want to use a specific estimator? You can do that. Want to modify the default settings for that estimator? You can also do that. Recipes just expand the number of One Model user personas able to leverage AI on their data. In Summary One AI Recipes provide a question-and-answer approach to building predictive models that answer key questions in the people analytics space. The resulting democratization of the production of AI models provides benefits including: Resource Savings Time Savings Interpretability Transparency Compliance Control and Flexibility You can have all of this as part of your people analytics platform by choosing One Model. Since you won’t learn about these Recipes by watching Food Network, schedule a demo here: Request a Demo
Read Article
Featured
5 min read
Stacia Damron
Is your company meeting its diversity goals? More importantly, if it is, are you adequately measuring diversity and inclusion success? While we may have the best intentions, today’s companies need to be focused on not just monitoring hiring metrics - but effectively analyzing them - in order to make a DE&I difference in the long term. But first, in order to do that, we need to take a look at key metrics for diversity and inclusion success. Let's talk about these diversity KPIs we’re measuring and why we’re measuring them. Without further ado, here’s 4 out-of-the box ways to measure diversity-related success that don’t have to do with hiring - all of which can help you supplement enhance your current reporting. Number 1: Rate and Timing of an Individual’s Promotions Are non-minority groups typically promoted every year and a half when minorities are promoted two years? Are all employees held accountable to the same expectations and metrics for success? Is your company providing a clearly-defined path to promotion opportunities, regardless of race or gender? Every hire should be rewarded for notable successes and achievement, and promoted according to a clear set of criteria. Make sure that’s happening across the organization - including minority groups. Digging into these metrics can help determine those answers and in the very least – put you on a path to asking the right questions. Number 2: Title and Seniority Do employees with the same levels of educational background and qualifications receive equitable salaries and titles? Often, minorities are underpaid compared to their non-minority counterparts. Measuring and tracking rank and pay metrics are two good ways to spot incongruences catch them early – giving your company a chance to correct a wage gap versus inadvertently widening it over time. Quantitative measures of diversity, like this, can help you see trends over time because changing diversity turning radius is a long process. Keep your eye on historically underpaid groups. A fairly paid employee is a happy, loyal employee. Number 3: Exposure to Upper Management and Inclusion in Special Assignments Global studies cited in a Forbes article revealed that a whopping 79 percent of people who quit their jobs cite ‘lack of appreciation’ as their reason for leaving. Do your employees – including minority groups - feel valued? Are you empowering them to make an impact? Unsurprisingly, people who feel a sense of autonomy and inclusion report higher satisfaction with their jobs – and are therefore more likely to stay. Are all groups within the organization equal-opportunity contributors? Bonus: On that note - are you performing any types of employee satisfaction surveys? Number 4: Training and Education Programs and Partnerships In 2014, Google made headlines for partnering with Code School. They committed to providing thousands of paid accounts to provide free training for select women and minorities already in tech. Does your company have a similar partnership or initiative with your community or company? As simple as it sounds – don’t just set it and forget it - track the relevant diversity KPIs that determine success and measure the results of your programs to determine if it is in fact, helping achieve your commitments towards improving diversity. The Summary: Success Comes by Measuring Diversity and Inclusion Hopefully, one of two (heck - maybe all four) of the items above resonated with you, and you’re excited to go tinker with your reporting platform. But wait - what if you have all this data, and you WANT to make some predictive models and see correlations in the data - and you’re all giddy to go do it - but you don’t have the tools in place? That’s where One Model can help. Give us your data in its messiest, most useless form, load it into our platform, and we’ll help you fully leverage that data of yours. Want to learn more? Let's connect about diversity metrics today. About One Model: One Model provides a data management platform and comprehensive suite of People Analytics directly from various HR technology platforms to measure all aspects of the employee lifecycle. Use our out-of-the-box integrations, metrics, analytics, and dashboards, or create your own as you need to. We provide a full platform for delivering more information, measurement, and accountability from your team.
Read Article
Featured
11 min read
Taylor Clark
The human resources department is a mission-critical function in most businesses. So the promise of better people decisions has generated interest in and adoption of advanced machine-learning capabilities. In response, organizations are adopting a wide variety of data science tools and technology to produce economically-optimal business outcomes. This trend is the result of the proliferation of data and the improved decision-making opportunities that come with harnessing the predictive value of that data. What are the downsides to harnessing machine learning? For one, machines lack ethics. They can be programmed to intelligently and efficiently drive optimal economic outcomes. It seems as though the use of machines in decisions and seemingly desirable organizational behaviors. Of course machines lack a sense of fairness or justice. But optimal economic outcomes do not always correspond to optimal ethical outcomes. So the key question facing human resources teams and technology support is "How can we ensure that our people decisions are ethical when a machine is suggesting those decisions?” The answer almost certainly requires radical transparency about how artificial intelligence and machine learning are used the decision making process. It is impossible to understand the ethical aspect of a prediction made by a machine unless the input data and the transformations of that data are clear and understood as well. General differences between various machine learning approaches have a profound impact on the ethicality of the outcomes that their predictions lead to. So let's begin by understanding some of those differences. Let’s focus on the various types of machine learning models: the black box model, the canned model, and the inductive model. What is a Black Box Model? A black box model is one that produces predictions that can’t be explained. There are tools that help users understand black box models, but these types of models are generally extremely difficult to understand. Many vendors build black box models for customers, but are unable or unwilling to explain their techniques and the results that those techniques tend to produce. Sometimes it is difficult for the model vender to understand its own model! The result is that the model lacks any transparency. Black box models are often trained on very large data sets. Larger training sets can greatly improve model performance. However, for this higher level of performance to be generalized many dependencies need to be satisfied. Naturally, without transparency it is difficult to trust a black box model. As you can imagine, it is concerning to depend on a model that uses sensitive data when that model lacks transparency. For example, asking a machine to determine if a photo has a cat in the frame doesn't require much transparency because the objective lacks an ethical aspect. But decisions involving people often have an ethical aspect to them. This means that model transparency is extremely important. Black box models can cross ethical lines where people decisions are concerned. Models, like humans, can exhibit biases resulting from sampling or estimation errors. They can also use input data in undesirable ways. Furthermore, model outputs are frequently used in downstream models and decisions. In turn, this ingrains invisible systematic bias into the decision. Naturally, the organization jeopardizes its ethical posture when human or machine bias leads to undesirable diversity or inclusion outcomes. One of the worst possible outcomes is a decision that is unethical or prejudicial. These bad decisions can have legal consequences or more. What is a Canned Model? The terms "canned model" or “off-the-shelf model” describe a model that was not developed or tailored to a specific user’s dataset. A canned model could also be a black box model depending on how much intellectual property the model’s developer is willing to expose. Plus, the original developer might not understand much about its own model. Canned models are vulnerable to the same biases as black box models. Unrepresentative data sets can lead to unethical decisions. Even a representative data set can have features that lead to unethical decisions. So canned models aren't without their disadvantages either. But even with a sound ethical posture, canned models can perform poorly in an environment that simply isn’t reflective of the environment on which the model was trained. Imagine a canned model that segmented workers in the apparel industry by learning and development investments. A model trained on Walmart’s data wouldn’t perform very well when applied to decisions for a fashion startup. Canned models can be quite effective if your workforce looks very similar to the ones that the model was trained on. But that training set is almost certainly a more general audience than yours. Models perform better when training data represents the real life population that was targeted and represented in the training set. What are Custom Built Models? Which brings us to custom built models. Custom models are the kind that are trained on your data. One AI is an example of the custom built approach. It delivers specialized models that best understand your environment because it’s seen it before. So it can detect patterns within your data to learn and make accurate predictions. Custom models discover the unique aspects of your business and learn from those discoveries. To be sure, it is common for data science professionals to deploy the best performing model that they can. However, the business must ensure that these models comply with high ethical and business intelligence standards. That's because it is possible to make an immoral decision with a great prediction. So for users of the custom built model, transparency is only possible through development techniques that are not cloudy or secret. Even with custom built models, it is important to assess the ethical impact that a new model will have before it is too late. Custom built models may incorporate some benefits of canned models, as well. External data can be incorporated into the model development process. External data is valuable because it can capture what is going on outside of your organization. Local area unemployment is a good example of a potentially valuable external data set. Going through the effort of building a model that is custom to your organization will provide a much higher level of understanding than just slamming a generic model on top of your data. You will gain the additional business intelligence that comes from understanding how your data, rather than other companies' data, relates to your business outcomes. The insights gleaned during the model development process can be valuable even if the model is never deployed. Understanding how any model performs on your data teaches you a lot about your data. This, in turn, will inform which type of model and model-building technique will be advantageous to your business decisions. Don’t Be Misled by Generic Model Performance Indicators A canned model’s advertised performance can be deceptive. The shape of the data that the canned model learned from may be drastically different from the data in your specific business environment. For example, if 5% of the people in the model's sample work remotely, but your entire company is remote, then the impact and inferences drawn by the model about remote work are not likely to inform your decisions very well. When to be Skeptical of Model Performance Numbers Most providers of canned models are not eager to determine the specific performance of their model on your data because of the inherent weaknesses described above. So how do you sniff out performant models? How can you understand a good smelling model from a bad smelling one? The first reason to be skeptical lies in whether the model provider offers relative performance numbers. A relative performance value is a comparative one, and therefore failing to disclose relative performance should smell bad. Data scientists understand the importance of measuring performance. They know that it is crucial to understand performance prior to using a model’s outputs. So avoiding relative performance, the vendor is not being 100% transparent. The second reason to be skeptical concerns vendors who can't (or won't) explain which features are used in their model and the contribution that each feature makes to the prediction. It is very difficult to trust a model's outputs when the features and their effects lack explanation. So that would certainly smell bad. One Model published a whitepaper listing the questions you should ask every machine learning vendor. Focus on Relative Performance….or Else! There are risks that arise when using data without relative performance. The closest risk to the business is that faith in the model itself could diminish. This means that internal stakeholders would not realize “promised” or “implied” performance. Of course, failing to live up to these promises is a trust-killer for a predictive model. Employees themselves, and not just decision makers, can distrust models and object to decisions made with it. Even worse, employees could adjust their behavior in ways that circumvent the model in order to “prove it wrong”. But loss of trust by internal stakeholders is just the beginning. Legal, compliance, financial, and operational risk can increase when businesses fail to comply with laws, regulations, and policies. Therefore, it is appropriate for champions of machine learning to be very familiar with these risks and to ensure that they are mitigated when adopting artificial intelligence. Finally, it is important to identify who is accountable for poor decisions that are made with the assistance of a model. The act of naming an accountable individual can reduce the chances of negative outcomes, such as bias, illegality, or imprudence. How to Trust a Model A visually appealing model that delivers "interesting insights" is not necessarily trustworthy. After all, a model that has a hand in false or misleading insights is a total failure. At One Model, we feel that all content generated from predictive model outputs must link back to that model's performance metrics. An organization cannot consider itself engaged in ethical use of predictive data without this link. Canned and black box models are extremely difficult to understand, and even more difficult to predict how they respond to your specific set of data. There are cases where these types of models can be appropriate. But these cases are few and far between in the realm of people data in the human resources function. Instead, custom models offer a much higher level of transparency. Model developers and users understand their own data much better throughout the model building process. (This process is called Exploratory Data Analysis, and it is an extremely under-appreciated aspect of the field of machine learning.) At One Model, we spent a long time -- more than 5 years -- building One AI to make it easier for all types of human resources professionals build and deploy ethical custom models from their data, while ensuring model performance evaluation and model explainability. One AI includes robust, deep reporting functionality that provides clarity on which data was used to train models. It blends rich discovery with rapid creation and deployment. The result is the most transparent and ethical machine learning capability in any people analytics platform. Nothing about One AI is hidden or unknowable. And that's why you can trust it. Their Artificial Intelligence Still Needs Your Human Intelligence Models are created to inform us of patterns in systems. The HR community intends to use models on problem spaces involving people moving through and performing within organizations. So HR pros should be able to learn a lot from predictive models. But it is unwise to relinquish human intelligence to predictive models that are not understood. The ultimate value of models (and all people analytics) is to make better, faster, more data-informed talent decisions at all levels of the organization. Machine learning is a powerful tool, but it is not a solution to that problem.
Read Article
Featured
10 min read
Joe Grohovsky
During my daily discussions with One Model prospects and customers, two consistent themes emerge: A general lack of understanding of predictive modeling and a delay in considering its use until basic reporting and analytical challenges are resolved. These are understandable, and I can offer a suggestion to overcome both. My suggestion is based upon seeing successful One Model customers gain immediate insights from their data by leveraging the technology found in our One AI component. These insights include data relationships that can surface even before customers run their first predictive model. Deeper insights before predictive modeling? How? To begin, let’s rethink what you may consider to be a natural progression for your company and your People Analytics team. For years we’ve been told a traditional People Analytics Maturity Continuum has a building block approach that is something like this: The general concept of the traditional People Analytics maturity model is based upon the need to master a specific step before progressing forward. Supposedly, increased value can be derived when each step is mastered, and the accompanying complexity is mastered. While this may seem logical, it is largely inaccurate in the real world. The sad result is many organizations languish in the early stages and never truly advance. The result is diminished ROI and frustrated stakeholders. What should we be doing instead? The short answer is to drive greater value immediately when your people analytics project launches. Properly built data models will immediately allow for basic reporting and advanced analytics, as well as predictive modeling. I’ll share a brief explanation of two One Model deliverables to help you understand where I’m going with this. People Data Cloud™️ Core Workforce data is the first data source ingested by One Model into a customer's People Data Cloud. Although additional data sources will follow, our initial effort is focused on cleaning, validating, and modeling this Core Workforce data.This analytics-ready data is leveraged in their People Data Cloud instance. Once that has occurred storyboards are then created, reflecting a customer’s unique metrics for reporting and analytics. It is now that customers can and should begin leveraging One AI (Read more about People Data Cloud). Exploratory Data Analysis One AI provides pre-built predictive models for customers. The capability also exists for customers to build their own bespoke models, but most begin with a pre-built model like Attrition Risk. These pre-built models explore a customer's People Data Cloud to identify and select relevant data elements from which to understand relationships and build a forecast. The results of this selection and ranking process are presented in an Exploratory Data Analysis (EDA) report. What is exploratory data analysis, you ask? It is a report that provides immediate insights and understanding of data relationships even before a model is ever deployed. Consider the partial EDA report below reflecting an Attrition Risk model. We see that 85 different variables were considered. One AI EDA will suggest an initial list of variables relevant to this specific model, and we see it includes expected categories such as Performance, Role, and Age. This first collection of variables does not include Commute Time. But is Commute Time a factor in your ideal Attrition Risk model? If so, what should the acceptable time threshold be? Is that threshold valid across all roles and locations? One AI allows each customer to monitor and select relevant data variables to understand how they impact insights into your predictive model. Changing the People Analytics Maturity Model into a Continuum Now that we realize that the initial Core Workforce People Data Cloud can generate results not only for Reporting and Analytics but also for Predictive Modeling, we can consider a People Analytics Maturity Continuum like this: This model recognizes the fact that basic reporting and analytics can occur simultaneously after a proper data lake is presented. It also introduces the concept of Monitoring your data and Understanding how it relates to your business needs. These are the first steps in Predictive Modeling and can occur without a forecast being generated. The truth underlining my point is: Analytics professionals should first understand their data before building forecasts. Ignoring One AI Exploratory Data Analysis insights from this initial data set is a lost opportunity. This initial model can and should be enhanced with additional data sources as they become available, but there is significant value even without a predictive output. The same modeled data that drives basic reports can drive Machine Learning. The greater value of One AI is providing a statistical layer, not simply a Machine Learning output layer. The EDA report is a rich trove of statistical data correlations and insights that can be used to build data understanding, a monitoring culture, and the facilitation of qualitative questions. But the value doesn’t stop there. Integrated services that accompany One AI also provide value for all data consumers. These integrated services are reflected in storyboards and include: Forecasting Correlations Line of BestFit Significance Testing Anomaly Detection These integrated services are used to ask questions about your data that are more valid than what can be derived solely from traditional metrics and dimensions. For example, storyboards can reflect data relationships so even casual users can gain early insights. The scatterplot below is created with Core Workforce data and illustrates the relationship between Tenure and Salary. One AI integrated services not only renders this view but cautions that based upon the data used this result is unlikely to be statistically significant (refer to the comment under the chart title below). More detailed information is contained in the EDA report, but this summary provides the first step in Monitoring and Understanding this data relationship. Perhaps one of the questions that may arise from this monitoring involves understanding existing gender differences. This is easily answered with a few mouse clicks: This view begins to provide potential insight into gender differences involving Tenure and Salary, though the results are still not statistically significant. Analysts are thus guided toward discovering their collection of insights contained within their data. List reports can be used to reflect feature importance and directionality. In the above table report, both low and high Date of Birth values increase Attrition Risk. Does this mean younger and older workers are more likely to leave than middle-aged workers? Interesting relationships begin to appear, and One AI automatically reports on the strength of those relationships and correlations. Iterations will increase the strength of the forecast, especially when additional data sources can be added. Leveraging One AI's capability at project launch provides a higher initial ROI, an accelerated value curve, and better-informed data consumers. At One Model, you don’t need to be a data scientist to get started with predictive modeling. Contact One Model to learn more and see One AI in action. Customers - Would you like more info on EDA reports in One Model? Visit our product help site.
Read Article
Featured
10 min read
Phil Schrader
Post 1: Sniffing for Bull***t. As a people analytics professional, you are now expected to make decisions about whether to use various predictive models. This is a surprisingly difficult question with important consequences for your employees and job applicants. In fact, I started drafting up a lovely little three section blog post around this topic before realizing that there was zero chance that I was going to be able to pack everything into a single post. There are simply no hard and fast rules you can follow to know if a model is good enough to use “in the wild.” There are too many considerations. To take an initial example, what are the consequences of being wrong? Are you predicting whether someone will click on an ad, or whether someone has cancer? In fact, even talking about model accuracy is multifaceted. Are you worried about detecting everyone who does have cancer-- even at the risk of false positives? Or are you more concerned about avoiding false positives? Side note: If you are a people analytics professional, you ought to become comfortable with the idea of precision and recall. Many people have produced explanations of these terms so we won’t go into it here. Here is one from “Towards Data Science”. So all that said, instead of a single, long post attempting to cover a respectable amount of this topic, we are going to put out a series of posts under that heading: Evaluating a predictive model: Good Smells and Bad Smells. And, since I’ve never met an analogy that I wasn’t willing to beat to death, we’ll use that smelly comparison to help you keep track of the level at which we are evaluating a model. For example, in this post we’re going to start way out at bull***t range. Sniffing for Bull***t As this comparison implies, you ought to be able to smell these sorts of problems from pretty far out. In fact, for these initial checks, you don’t even have to get close enough to sniff around at the details of the model. You’re simply going to ask the producers of the model (vendor or in-house team) a few questions about how they work to see if they are offering you potential bull***t. At One Model, we're always interested in sharing our thoughts on predictive modeling. One of these great chats are available on the other side of this form. Back to our scheduled programming. Remember that predictions are not real. Because predictive models generate data points, it is tempting to treat them like facts. But they are not facts. They are educated guesses. If you are not committed to testing them and reviewing the methodology behind them, then you are contenting yourself with bull***t. Technically speaking, by bull***t, I mean a scenario in which you are not actually concerned with whether the predictions you are putting out are right or wrong. For those of you looking for a more detailed theory of bull***t, I direct you to Harry G. Frankfurt. At One Model we strive to avoid giving our customers bull***t (yay us!) by producing models with transparency and tractability in mind. By transparency we mean that we are committed to showing you exactly how a model was produced, what type of algorithm it is, how it performs, how features were selected, and other decisions that were made to prepare and clean the data. By tractability we mean that the data is traceable and easy to wrangle and analyze. When you put these concepts together you end up with predictive models that you can trust with your career and the careers of your employees. If, for example, you produce an attrition model, transparency and tractability will mean that you are able to educate your data consumers on how accurate the model is. It will mean that you have a process set up to review the results of predictions over time and see if they are correct. It will mean that if you are challenged about why a certain employee was categorized as a high attrition risk, you will be able to explain what features were important in that prediction. And so on. To take a counter example, there’s an awful lot of machine learning going on in the talent acquisition space. Lots of products out there are promising to save your recruiters time by using machine learning to estimate whether candidates are a relatively good or a relatively bad match for a job. This way, you can make life easier for your recruiters by taking a big pile of candidates and automagically identifying the ones that are the best fit. I suspect that many of these offerings are bull***t. And here are a few questions you can ask the vendors to see if you catch a whiff (or perhaps an overwhelming aroma) of bull***t. The same sorts of questions would apply for other scenarios, including models produced by an in-house team. Hey, person offering me this model, do you test to see if these predictions are accurate? Initially I thought about making this question “How do you” rather than “Do you”. I think “Do you” is more to the point. Any hesitation or awkwardness here is a really bad smell. In the talent acquisition example above, the vendor should at least be able to say, “Of course, we did an initial train-test split on the data and we monitor the results over time to see if people we say are good matches ultimately get hired.” Now later on, we might devote a post in this series to self-fulfilling prophecies. Meaning in this case that you should be on alert for the fact that by promoting a candidate to the top of the resume stack, you are almost certainly going to increase the odds that they are hired and, thus, you are your model is shaping, rather than predicting the future. But we’re still out at bull***t range so let’s leave that aside. And so, having established that the producer of the model does in fact test their model for accuracy, the next logical question to ask is: So how good is this model? Remember that we are still sniffing for bull***t. The purpose of this question is not so much to hear whether a given model has .75 or .83 precision or recall, but just to test if the producers of the model are willing to talk about model performance with you. Perhaps they assured you at a high level that the model is really great and they test it all the time-- but if they don’t have any method of explaining model performance ready for you… well… then their model might be bull***t. What features are important in the model? / What type of algorithm is behind these predictions? These follow up questions are fun in the case of vendors. Oftentimes vendors want to talk up their machine learning capabilities with a sort of “secret sauce” argument. They don’t want to tell you how it works or the details behind it because it’s proprietary. And it’s proprietary because it’s AMAZING. But I would argue that this need not be the case and that their hesitation is another sign of bull***t. For example, I have a general understanding of how the original Page Rank algorithm behind Google Search works. Crawl the web and work out the number of pages that link to a given page as a sign of relevance. If those backlinks come from sites which themselves have large numbers of links, then they are worth more. In fact, Sergey Brin and Larry Page published a paper about it. This level of general explanation did not prevent Google from dominating the world of search. In other words, a lack of willingness to be transparent is a strong sign of bull***t. How do you re-examine your models? Having poked a bit at transparency, these last questions get into issues of tractability. You want to hear about the capabilities that the producers of the model have to re-examine the work they have done. Did they build a model a few years ago and now they just keep using it? Or do they make a habit of going back and testing other potential models. Do they save off all their work so that they could easily return to the exact dataset that was used to train a specific version of the model. Are they set up to iterate or are they simply offering a one-size fits all algorithm to you? Good smells here will be discussions about model deployment, maintenance and archiving. Streets and sewers type stuff as one of my analytics mentors likes to say. Bad smells will be high level vague assurances or -- my favorite -- simple appeals to how amazingly bright the team working on it is.If they do vaguely assure you that they are tuning things up “all the time” then you can hit them with this follow up question: Could you go back to a specific prediction you made a year ago and reproduce the exact data set and version of the algorithm behind it? This is a challenging question and even a team fully committed to transparency and tractability will probably hedge their answers a bit. That’s ok. The test here not just about whether they can do it, but whether they are even thinking about this sort of thing. Ideally it opens up a discussion about you they will support you, as the analytics professional responsible for deploying their model, when you get challenged about a particular prediction. It’s the type of question you need to ask now because it will likely be asked of you in the future. As we move forward in this blog series, we’ll get into more nuanced situations. For example, reviewing the features used in the predictions to see if they are diverse and make logical sense. Or checking to see if the type of estimator (algorithm) chosen makes sense for the type of data you provided. But if the model that you are evaluating fails the bull***t smell test outlined here, then it means that you’re not going to have the transparency and tractability necessary to pick up on those more nuanced smells. So do yourself a favor and do a test whiff from a ways away before you stick your nose any closer.
Read Article