Codesmith Outcomes Reporting: A Conversation with James White of Banks, Finley, White & Company

Transparent employment outcomes empower our community to make the best possible decisions about their education and career. That's why we prioritize graduate outcomes reporting through the Council on Integrity in Results Reporting (CIRR) – the industry standard for trustworthy results reporting. 

CIRR was established in 2016 as a 501(c)(6) non-profit to set shared standards on reporting student outcomes, focused entirely on reporting accurate and transparent outcomes that are verified by an American Institute of Certified Public Accountants (AICPA) peer reviewed auditing firm. With CIRR reports, folks considering a coding program can see schools’ graduation rates, the percentage of alumni employed in the field, median salary out of the program, and more.

An important aspect of CIRR reporting is the auditing process. We sat down with James White, Managing Partner at AICPA peer reviewed auditing firm Banks, Finley, White & Company, to discuss how the firm ensures Codesmith’s CIRR outcomes reports are accurate and follow the strict guidelines set forth by CIRR.

What is an auditor, and what do they do?

An auditor, in its most base form, is someone who conducts assessments of processes and systems, and reviews information to conform to a body of either policies or procedures to make sure that that information conforms to that and is valid in association with those policies and procedures. 

As a Certified Public Accountant, we're a self-regulated body governed by the American Institute of Certified Public Accountants, or AICPA. As a self-regulated body, we dictate all of those key cornerstones of [an audit] – independence, ethics, integrity, and making sure that those things are performed correctly.

As the auditor for Codesmith's CIRR (Council on Integrity in Results Reporting) employment outcomes reports, what types of data are you auditing?

The employment outcomes reporting is special in the sense that it doesn't typically relate to [an organization’s] financial information. So, our first step in looking at the data as relates to employment outcomes is to look at the rules as prescribed by CIRR. First and foremost, as auditors, we need to understand what it is that Codesmith, or any other coding school, is supposed to be reporting on. That's paramount so that you can understand what could go wrong, what we are looking for as it relates to that information, and then what needs to be validated to make sure that Codesmith meets those standards.

Once we've done that, and developed our game plan, the next thing we look at is the detailed compilation of all of the data as it relates to CIRR. This primarily involves the school reaching out to all of its past students, and confirming their employment status over a 90 and 180 day period to validate some key pieces of information – (a) are they employed and (b) what was their salary inside of a certain range? And, naturally, the school obtains some validation of that information, be it from a confirmation from them directly, or some other validation of that information. 

How do you make sure that Codesmith’s outcomes data is accurate and reliable?

Our primary procedure and step is to independently test the information. We reach out to the former students directly and independently, and ask them to confirm some key pieces of information. That is the best evidence we can obtain, because we've had that direct communication for them to validate what is being reported and if it is correct, or, if there is something that may be different now.

Differences arise from time to time. Maybe, depending upon the amount of time in which we reach out to a former student, they've gotten a salary increase. That's an explainable difference. There's steps to follow after that. Our primary step is to confirm independently with the former student. Then, secondarily, there's publicly available information. In this day and age, LinkedIn is almost as gospel as anything else. So we use that to help us validate and verify as much as we possibly can.

Can you walk us through the auditing process for Codesmith’s employment outcomes data?

First and foremost, we confirm the CIRR policies and procedures to make sure that those haven't changed from the previous cohorts’ report. Then, once we've obtained the full list of data, in detail and in the summarized form that's presented to CIRR for posting publicly, we will reperform the determination of that data. We'll take all the data, and basically recreate it ourselves to ensure that those two things match up.

Then, we'll make a generally very large selection of former students across those different classifications within the summary reporting, and we'll send out confirmations to them. Unfortunately, we will have to hit them one, two, or three times. But, it's all in the goal of ensuring that what the company is posting out there publicly – information that people very well may be relying upon as they make school choices – is valid, accurate, and reasonable for people to use to make any decisions that they may make. 

In the event there are some differences, we'll follow up first with the former student we confirmed with to ask them for any clarification or understanding. Then, we'll have a discussion with the company about the differences to say, ‘Here's what they reported. Here's what you had. Let's understand why there was a difference.’ It may be indicative of an issue in the process, right? Maybe we need to change how you guys are asking for the information. Or, maybe there was a typo. These things do happen from time to time.

So, if we come to the conclusion that it’s not a widespread event, at the end of all that we issue a report that basically says that the information that's contained within the CIRR report, as posted, is accurate, reasonable, and you can rely upon it based on the procedure that we performed. We provide that on our letterhead to the company to post along with their report that is posted on the CIRR website as well.

How do auditors maintain independence and objectivity when auditing employment outcomes data?

As we're self-regulated and we hold our profession in high regard, independence and integrity are paramount in the profession from that perspective. So, naturally, there are standards and guidelines that prescribe what we're able to do as relates to ourselves and Codesmith so that both in fact and in appearance we are independent of the company. 

Why is the work of auditing employment outcomes from coding schools, like Codesmith, so important? And, how does it contribute to the credibility and transparency of those organizations?

At the end of the day, [the audit] is paramount. [Auditing] all started with people putting out financial information, and people saying, ‘we need somebody to go and validate this information because I'm making financial decisions based on the information that's out there.’ And, a number can be changed simply by a keystroke. The core behind the accounting profession is to help instill trust in the public markets by providing a resource for people to go and say, these people did their job and we're able to rely upon this information.

I think it translates very seamlessly to the kind of information that you are putting out about Codesmith, or any coding school. You're giving the results of not just the quality of the education, but how the company is viewed by employers out there in the market. So, people are going to make decisions as to which school they should choose [based on that information]. If a story is being spun because that information has been tweaked, then people are going to make life decisions based on bad data. That can be life changing, sometimes for the good, but most of the time for the negative. That's why it's so paramount that if information is being put out for people to make decisions upon, it's a base requirement for that information to be verified by somebody that has nothing to do with it. That's what the accounting profession is here to do. 

Check out our CIRR FAQ for more information on CIRR reports and data collection. To view Codesmith’s latest CIRR outcomes reports, visit https://cirr.org/data.