Join our discussion on creating HR analytics data integration architecture between common HRIS systems including workday, successfactors, oracle, adp.
This vendor-by-vendor comparison will be a living post and we will continue to update as we have time to collect thoughts on each vendor and as we complete integrations with new vendors. Not every source we work with will be listed here but we'll cover the major ones that we often work with.
At One Model we get to see the data and structure from a load of HR systems, and beyond, basically anything that holds employee or person data is fair game as a core system to integrate for workforce analytics.
After more than a decade of HR analytics integration architecture experience where the solution is directly integrating data from these systems into analytics and reporting solutions, we have a lot of experience to share.
Below I'll share our experience with highlights from each system and how they align with creating a people analytics warehouse. Some are better than others from a data perspective and there's certainly some vendors that are yet to understand that access to data is already a core requirement of buyers looking at any new technology.
Bookmark this blog, add your email to the subscription email list to the right, or follow me (Chris Butler) and One Model on LinkedIn to stay up to date.
Ratings are provided as an anecdotal and unscientific evaluation of our experience in gaining access to, maintaining, and working with the data held in the associated systems. They are my opinions.]
If you would like to make use of any of our integrations in a stand-alone capacity, we now offer a data warehouse only product where you utilize just our data pipeline and modelling engine to extract and transform data into a data warehouse hosted by One Model or your own data warehouse. We'll be releasing some more public details soon but you are a company that likes to roll your own analytics, visualizations, and just need some help with the data side of the house, we can certainly help. Contact Us
Workday analytics embedded into the product is underwhelming and we're yet to see Prism Analytics make a dent in filling the needs that people analytics teams or HR analysts have beyond convenience analytics. So in the meantime, if you are serious about improving reporting and people analytics for Workday you're going to need to get the data out of there and into somewhere else.
On the surface, Workday looks to have a great API, and the documentation available is excellent. However, the single biggest downfall is that the API is focused on providing a snapshot, which is fine for simple list reports but does not allow a people analytics team to deliver any worthwhile historical analysis. You don't get the bulk history output of other systems or the ability to cobble it together from complete effective-dated transactions across objects. To capture the complete history we had to build an intense process of programmatically retrieving data, evaluating, and running other API calls to build the full history that we need. If you want more detail take a look at my blog post on the subject The end of the snapshot workday edition.
The complexity of the integration, therefore, is multiplied and the time taken suffers immensely due to the object-oriented architecture that requires you to load each object into memory in order to be able to retrieve it. A full destructive data extraction means you're looking at 8+ hours for a small-medium enterprise and expanding to a week if you're a giant. The problem is exacerbated by the number of parallel connections allowed to run at a fraction of the stated limit. A full historical API integration here is not for the faint of heart or skill, we have spent 12+ months enhancing and tweaking our integration with each release (weekly) to improve performance and solve data challenges. Our integration to give a sense of scale generates some 500+ tables that we bring together in our modelling engine in preparation for analytics.
Out-of-the-box integration plugins are going to be focused on the snapshot version of data as well so if you don't have the integration resources available I wouldn't attempt an API integration. My advice is to stick with the built-in reporting tools to get off the ground. The RaaS tools do a good job of combining objects and running in a performant manner (better than the API). However, they will also be snapshot focused and as painful as it will be to build and run each timepoint you will at least be able to obtain a basic feed to build upon. You won't have the full change history for deeper analysis until you can create a larger integration, or can drop in One Model. Robert Goodman wrote a good blog a little while back looking at both the API and his decision to use RaaS at the time, take a read here. Workday API vs RaaS
Regardless of the problems we see with the architecture, the API is decent and one of our favorite integrations to work with. It is, however, little wonder that with the data challenges we have seen and experienced, half of our customers are now Workday customers.
One Model consumes the Public Web Service API's for all standard objects and fields. One Model configures and manages the services for API extractions, customers need only to create and supply a permissioned account for the extraction.
Custom objects and fields need to use a Raas (Report as a service) definition created by the customer in the Enterprise Interface Builder (EIB). The Report can then be transferred by SFTP or can be interacted with as an API itself.
Figure 1: One Model's data extraction from Workday
4 out of 5 is a pretty phenomenal rating in my book. I almost gave SuccessFactors a perfect 5 but there are still some missing pieces from the API libraries and we've experienced some dropped data at times that have required some adaptations in our integration. Overall, the collection of SF APIs is a thing of beauty for one specific reason: it is dynamic and can accommodate any of the Meta Data Framework (MDF) custom changes in its stride. This makes life incredibly easy when working across multiple different customers and means we can run a single integration against any customer and accurately retrieve all customizations without even thinking about them. Compared to Workday where the API is static in definition and only covers the standard objects this facet alone is just awesome.
This dynamic nature though isn't without its complexities. It does mean you need to build an integration that can interrogate the API and iterate through each of its customizations. However, once it is complete it functions well and can adapt to changing configurations as a result.
Multiple API endpoints also require different integrations to be merged. This is a result of both upgrades in the APIs available in the case of the older SuccessFactors API and the OData API as well as providing an API to acquired parts of the platform (i.e. Learning from the Plateau acquisition). We're actually just happy there is now an API to retrieve learning data as this used to be a huge bug bear when I worked at SuccessFactors on the Workforce Analytics product. The only SF product I know of right now that doesn't have the ability to extract from an API is Recruiting Marketing (RMK) from the jobs2web acquisition, hopefully, this changes in the future.
Full disclosure, I used to hate working with SuccessFactors data when we had to deal with flat files and RDFs, but with the API integration in place, we can be up and running with a new SuccessFactors customer in a few hours and be confident all customizations are present.
I haven't spoken here about the new Integration Center release from earlier last year as we haven't used it ourselves and only have anecdotal evidence from what we've read. It looks like you could get what you need using the Integration Center and deliver the output to your warehouse. You will obviously need to build each of the outputs for the integration which may take a lot of time but the data structure from what I can tell looks solid for staging into an analytics framework. There are likely a lot of tables to extract and maintain though, we currently run around 400+ tables for a SuccessFactors customer and model these into an analytics-ready model. If anyone has used the Integration Center in an analytics deployment please feel free to comment below or reach out and I would be happy to host your perspective here.
One Model consumes the SF REST API's for all standard fields as well as all customized fields including any use of the MDF framework. One Model configures and manages the service for API extractions, customers need only to create and supply a permissioned account for the extraction.
SF has built a great API that is able to provide all customizations as part of the native API feed. We do us more than one API though as the new OData API doesn't provide enough information and we have to use multiple endpoints in order to extract a complete data set. This is expertly handled by One Model software.
Figure 2: One Model's data extraction from SuccessFactors
I think I lost a lot of hair and put on ten pounds (or was it ten kilos?!) working through a suitable extraction method for the HCM Cloud suite that was going to give us the right level of data granularity for proper historically accurate people analytics data. We tried every method of data extraction from the API to using BI Publisher reports and templates. I can see why people who are experienced in the Oracle domain stick with it for decades, the experience here is hard-won and akin to a level of magic. The barriers to entry for new players are just so high that even I as a software engineer, data expert, and with a career spent in HR data many times over, could not figure out how to get a piece of functionality working that in other systems would take a handful of clicks.
In looking to build an extraction for people analytics you have a number of methods at your disposal. There's now an API and the built-in reporting could be a reasonable option for you if you have some experience with BI Publisher. There are also the HCM Extracts built for bulk extraction purposes. We quickly discounted the API as not yet being up to scratch for people analytics purposes since it lacks access to subject areas, and fields, and cannot provide the level of history and granularity that we need. I hope that the API can be improved in the future as it is generally our favorite method for extraction. We then spent days and probably weeks trying to get the built-in reporting and BI Publisher templates to work correctly and deliver us the data we're used to from our time using Oracles on-premise solutions (quite a good data structure). Alas, this was one of the most frustrating experiences of my life, it really says something when I had to go find a copy of MS Word 2006 in order to use a plugin that for some reason just wouldn't load in MS Word 2016, all to edit and build a template file to be uploaded, creating multiple manual touchpoints whenever a change is required. Why is life so difficult??
Even with a bunch of time lost to this endeavour our experience was that we could probably get all the data we needed using the reporting/BI publisher route but that it was going to be a maintenance nightmare if an extract had to change requiring an Oracle developer to make sure everything ran correctly. If you have experienced resources this may work for you still.
We eventually settled on the HCM Extracts solution provided that while mind-numbingly frustrating to use the interface to build and extract will at least reliably provide access to the full data set and deliver it in an output that with some tooling can be ingested quite well. There are a number of options for how you can export the data and we would usually prefer a CSV style extraction but the hierarchical nature of the extraction process here means that XML becomes the preferred method unless you want to burn the best years of your life creating individual outputs for each object tediously by hand in a semi-responsive interface. We, therefore, figured it would be easier, and enhance maintainability if we built our own .xml parser for our data pipeline to ingest the data set. There are .xml to .csv parsers available (some for free) if you need to find one but my experience with them is they struggle with some files to deliver a clean output for ingestion.
With an extract defined though there's a good number of options on how to deliver and schedule the output and reliability is good. We've only had a few issues since the upfront hard work was completed. Changing an extract as well is relatively straightforward if you want to add a field or object you can do so through the front-end interface in a single touchpoint.
We do love Oracle data, and don't get me wrong - the construction and integrity are good and we have a repeatable solution for our customer base that we can deliver at will, but it was a harrowing trip of discovery that to me, explains why we see so few organizations from the Oracle ecosystem that are out there talking about their achievements. Don't make me go back, mommy!
Chris Butler : Mar 29, 2017 12:00:00 AM
I recently made a simple post on LinkedIn which received a crazy amount of views and overwhelmed us with requests to take a look at what we had...
Chris Butler : Jun 1, 2020 12:00:00 AM
The first in a series of posts tackling the individual nuances we see with HR technology systems and the steps we take in overcoming their native...
Chris Butler