What Magneto's Helmet and Non-ICFR SSAE 16 Controls have in Common


Both are designed to mitigate unlikely risks.  For anyone who may not know, Magneto's nemesis, Professor X, has the ability to read minds, and in order to protect his thoughts from being intercepted, Magneto wears a helmet made of lead.  Magneto and Professor X are not real, and neither is the ability to read minds, but judging by many of the controls I see in SSAE 16 reports that are supposed to contain only controls that mitigate the risk of material financial misstatement, I would not be surprised to see a control such as "We use lead-lined drywall in our offices to prevent psychics from seeing us enter our passwords."

Recently I have had opportunities to observe several auditors defend why they believe the controls contained in their client's SSAE 16 reports are relevant to internal controls over financial reporting (ICFR).  The auditors argue that the risk events the controls mitigate are "possible."  I have had to explain, to seasoned auditors in some cases, that just because a risk event is possible, does not mean it is reasonable to conclude a control is ICFR relevant.  I mean, sure, some people believe in psychics, and there is probably anecdotal evidence out there that password sniffing via psychic is "possible," but is jumping from there to a control requirement for lead-lined drywall reasonable?  Of course not.  It is equally unreasonable to say that having fire suppression in your data center will prevent you from materially misstating your financials.

Do we really need to explain that physical security, environmental, and operational controls do not belong in SSAE 16 reports, or is there something more sinister going on?  Why haven't auditors required their clients to remove non-ICFR controls from their SSAE 16 reports as the standard requires for the past two years since the launch of SOC reports?  Could it be that the number of controls that require testing in an audit program directly relate to the audit fees that are justified?  

I remember that in years three or four of Sarbanes Oxley compliance my clients would start "control rationalization exercises" where they would evaluate the risk of a control, in order to justify its removal from the external auditor's scope.  It was easy for management to calculate the value of performing this exercise...total audit fees, divided by the number of controls, equals audit fee per control.  It stood to reason that the more controls you could "rationalize", the lower the audit fee should be.

I believe that most auditors know that it is unreasonable to include physical security, environmental, and operational controls in SSAE 16 reports, but they are unwilling to do anything about it until their clients sign up for SOC 2 engagements.  As long as clients do that, audit fees will increase rather than decrease.  How short sighted is that though?  Requiring clients to remove non-ICFR controls would have created an assurance vacuum that SOC 2 was designed to fill.  As it stands now, auditors who issued non-compliant SSAE 16 reports have a credibility problem.

If you are hearing about SOC 2 for the first time, and are looking to engage an auditor, you should ask to see some of their SSAE 16 work before making that decision.  If you are considering a report other than SOC 2 for assurance pertaining to security, availability, and/or confidentiality, please read my posts on why SOC 2 is a superior report to all others below:  

SOC 2 is being Ignored by the Data Center Industry

On the anniversary of SOC report launch,  of Reckenen Accountants & Associates published the following results of their survey of data centers in the U.S.  The results clearly indicate the market's response to the AICPA's attempt to correct the misuse of SAS 70 that was so pervasive in the industry before SSAE 16, and that continues under the new attestation standard.

With the rejection of SOC 2, the data center industry is telling the AICPA, we hear you, but we are not interested.  Our customers are not asking for SOC 2 because our SOC 1 (SSAE 16) reports contain the information they are looking for with regard to the security, availability, processing integrity, confidentiality, and privacy of the services we provide them.

Until the AICPA gets tough on CPAs that are issuing opinions under the SSAE 16 attestation standard on engagements that include testing of controls that are not relevant to internal controls over financial reporting (non-ICFR controls), there will not be a significant change in this reality.  Two years after the launch of SOC reports, the data center industry is ignoring SOC 2.

Celebrating SOC’s 2nd Birthday (Infographic) - An Infographic from SOC Audit Examinations & Outsourced Accounting for Private Companies

Hosting.com is in the Elite Group of CSPs with SOC 2 Reports

I have been observing the transition of SAS 70 to SOC reports over the past two years, and have noted that the same misuse that was occurring with SAS 70 reports carried forward to SOC 1  (SSAE 16) reports.  I would be remiss, however if I did not highlight cloud service providers (CSPs) that have chosen to undergo the more applicable and rigorous SOC 2 attestation engagement.  Based on my observations, Hosting.com is among the Top 10% in SOC compliance rigor – not only keeping up with key details behind compliance – but actually ensuring that compliance is a part of their solutions, so their customers have less to worry about.

In the embedded webinar below, Sean Bruton of Hosting.com explains why they decided to pursue a SOC 2.  Here are a few of his note worthy quotes:

2:18 “SAS 70 has a long history of having nothing to do with IT service providers.”

4:10 “SSAE 16 retains the original purpose of SAS 70.  That is, compliance around internal controls over financial reporting.”

5:18 Gartner Weighs In – SAS 70 is useless for security

5:35 Now there are options.

6:08 “The AICPA is saying that SOC 1 is the wrong report for IT service providers to run.”

6:45 “If your service provider has passed a SOC 2 assessment; if you got a SOC 3 report, then what you have is assurance that they met a minimum standard of due diligence.”


7:30 “SOC 3 is great for marketing purposes, but get a copy of the [SOC 2] report!  The reason you do this is because at the end of the day, [without the detail a SOC 2 report provides], you have no knowledge of specifically, what controls the service provider has implemented, what is ultimately your responsibility to implement with the applications you are hosting in the cloud.”

Sean is an extremely rare individual, and I hope to be able to interview him in person very soon.  Connect with him on LinkedIn here:

Irregardless, Begs the Question, and SSAE 16 Certified

These are words that resemble the sound of nails scratching a chalkboard to me.  "Irregardless" is not a word, and is not a substitute for irrespective or regardless.  "Begging the question" is a logical fallacy, not a substitute for "...which raises the question...", and there is no such thing as an "SSAE 16 certification".

For the past two years, I have supported the AICPA's efforts to correct the misuse of SAS 70 by replacing it with SOC reports, yet day after day I read press releases and blog posts by companies claiming that their SSAE 16 certification proves that their services are secure and available.

I think I may have finally begun to realize the futility of it all though.  We used to say, "Ain't ain't a word, because it ain't in the dictionary", but that's not the case anymore.  It's there!  It has just been labeled "non-standard."  Just as widespread use of "ain't" and irregardless have led to them being added to the dictionary, maybe it's time to just label the misuse of SSAE 16 reports as "non-standard" and let it go.

What about serving the public interest though?  The Code of Professional Conduct says: A distinguishing mark of a profession is acceptance of its responsibility to the public (Rule 201 Section ET 53 – Article II – The Public Interest).  What about the customer of an outsourcing vendor who sees a fake SSAE 16 logo, reads that the company they are doing business with has been "SSAE 16 certified", and proceeds to place reliance on a report that the AICPA says is not designed to provide assurance regarding security or availability?  If the CPA firm who issued the SSAE 16 report does not disassociate themselves from such a company, and if the AICPA does not hold them accountable for doing so, then has the public interest been served?

Calling the report a certification is only part of the problem though.  This slide from an AICPA presentation (that you can download by clicking on it), says that SAS 70 reports contained controls related to subject matter other than internal control over financial reporting (ICFR).  That problem persists today...two years after SOC reports replaced SAS 70.

We cannot really blame service organizations or their customers for thinking a report containing environmental and operational controls, tested by an independent CPA firm, provides assurance about the security and availability of their services though can we?  After all, what's wrong with relying on my data center's SSAE 16 report if I need to know that they have a diesel generator for backing up commercial power in case there is a power outage, and the report includes that testing?  The same thing goes for having UPS units, fire extinguishers, raised flooring, etc.

The problem is that these things have nothing to do with my data center's role in assuring the accuracy of my financial statements, and they are not supposed to be included in the report.  To comply with their professional standards, every CPA must require their clients to remove non-ICFR controls.  Yet two years after the launch of SOC reports, every SSAE 16 report I have seen contains non-ICFR controls, and the auditor has issued an opinion as to their effectiveness.  I have seen guidance from CPA firms that list removal of these kinds of controls as optional, and have clients who tell me their CPA firm has never even mentioned the need for a re-evaluation of their controls for ICFR applicability.

At the risk of sounding like the annoying guy who corrects people's use of the word irregardless, I will say the following:


  • If you are a company relying on service providers, and your service provider gives you their SSAE 16 report as assurance that their services are secure and available, demand a SOC 2 report, or walk away.  
  • If you are a service provider, and your CPA firm has not walked you through re-evaluation of your controls for ICFR applicability, contact me, or a CPA firm that will help you through that process.  
  • If you are a CPA firm who has clients who still want to include blatantly non-ICFR controls in their SSAE 16 reports, then have the courage to say you will not opine on them this year, and that they must be moved to the other information section.

Let us all do our part to stop the misuse of SSAE 16 reports.


Thank you for the endorsement French.  It is indeed an honor to have the ear of the Gartner Risk Management analyst who first reported SAS70 abuse in 2010.

Want to Talk about Compliance Sometime?

Mountain View, California...I hear you.  If you ever want to chat, you know how to reach me.


Same thing goes for Foster City, California.  I'd love to chat sometime.


Amazon Web Services (AWS) Completes SOC 2 and SOC 3 Engagement

The list scope of services that the engagement covered included Amazon EC2, Amazon S3, Amazon VPC, Amazon EBS, Amazon RDS, Amazon DynamoDB, Amazon SimpleDB, Amazon Direct Connect, Amazon VM Import/Export, Amazon ElastiCache, Amazon Glacier, Amazon Storage Gateway, Amazon Elastic MapReduce, Amazon Redshift and Amazon Identity and Access Management and their supporting data centers, including the GovCloud (US) region.

Here's a link to Chad Woolf, Director of AWS Risk and Compliance discussing it in his blog.

This ought to light a fire under anyone out there still using their SOC 1 (SSAE 16) report to provide assurance that it is not designed to provide.

SOC 2 Can Be Relied on in Financial Statement Audits


UPDATE:  The guidance has been taken down.  Follow the link here...  Let's see what happens next.

This is big news!  The AICPA said the following in guidance released today:


This means that those service organizations who were left scratching their heads when their publicly traded customers told them that they would need a SOC 1 report that covers physical security because their external auditors had told them that they would get a significant deficiency on their financial statement audit otherwise, even though their SOC 2 covers physical security in more depth can breathe a sigh of relief.

SOC 2 Reports Replaced SAS 70 for Security and Availability in 2011 - This is 2013.

The market is making progress towards correcting the notion that SAS 70 or SSAE 16 provide assurance regarding security, but last week I came across a service organization's publication that included the following statement:


What is the difference between SSAE16 SOC-1 and SOC-2?

SOC 2 reports will contain the same report elements as SOC 1 reports but will be prepared in accordance with the AT Section 101 attest standard rather than the SSAE16 standard. Furthermore, the control objectives in a SOC 2 report will be based on the AICPA and CICA’s Trust Service Principles and Criteria, previously used by the WebTrust and SysTrust certifications. Like SOC 1 reports, SOC 2 reports are available in a Type 1 and a Type 2 report. [We are] evaluating SOC-2 reports but [are] not currently actively engaged in SOC 2 audits.

Stop here for a minute and think about the issues in this paragraph.

SOC 2 reports will contain...
SOC 2 is placed into a "future" category instead of something that should have already been implemented.  We are, after all, approaching two years since the AICPA prohibited the inclusion of non-ICFR controls in SOC 1 reports to curb the misuse of SAS 70 as a report that provided assurance regarding security.  SOC 2 reports replaced SAS 70 for security and availability in 2011.

SOC 2 reports will contain the same report elements as SOC 1 reports but will be prepared in accordance with the AT Section 101 attest standard rather than the SSAE16 standard.
Although technically correct because SOC 2 includes the same report format (elements) as a SOC 1 report, the sentence infers that the content of SOC 2 and SOC 1 are essentially the same to the uniformed reader.

Furthermore, the control objectives in a SOC 2 report will be based on the AICPA and CICA’s Trust Service Principles and Criteria, previously used by the WebTrust and SysTrust certifications.
The sentence again uses future context, and also makes sure to include a reference to WebTrust and SysTrust to associate SOC 2 with the reports ignored in large part by the market.  SOC 2 is the AICPA's second attempt at getting people to take non-ICFR controls out of reports that are focused on supporting financial statement assertions.

[We are] evaluating SOC-2 reports but [are] not currently actively engaged in SOC 2 audits.
Translation: "We are waiting to see if the market will really require us to undergo one of these engagements that cost more, and require greater scrutiny of our internal controls as they relate to security and availability."

Customers need to demand risk assurance pertaining to security and availability from their vendors.  The proper report for this type of assurance is a SOC 2 report.  If your vendor is asking you to be content with a SOC 1 (SSAE 16) report that the AICPA and Gartner have made abundantly clear does not provide that assurance, then red flags should go up about that vendor.

Which is better: SOC 1 or SOC 2?

I have been asked this question by several current and prospective clients.  My latest response was this- "Which are better: skates or skis?"

Ice skates are just as useless on snow as snow skis are on ice.  In the same way SOC 1 is useless for providing assurance regarding security and availability because it is designed for supporting financial statement assertions.  SOC 2 is equally useless for providing assurance relevant to internal controls over financial reporting (ICFR) because it is designed to provide assurance regarding security, availability, processing integrity, confidentiality, and privacy.  Neither is better than the other, they are just designed for different purposes.

Why then, do we continue to see press releases that tout SOC 1 (SSAE 16) as providing assurance regarding security and availability?  The Risk Assurance Guy and French Caldwell at the Gartner Group seem to be the only ones pointing out that this abuse continues in the market.


Watch people trying to ski with ice skates here:

Data Centers with SOC 2 Reports Stand Out

Congratulations to Reliable Networks on their completion of a SOC 2 engagement.  Great press release too!

How to Explain SOC 2 to Your Clients Who Are Still Asking for SSAE 16

If you have made the decision to go with a SOC 2 because it is the right report for your organization, but you still have customers who are asking for a SOC 1 (SSAE 16), here's what you can tell them.

Dear Customer, 

We have determined through careful examination of our services, and in consultation with our CPA firm that SOC 1 (SSAE 16) is not applicable to us because our services do not have the potential to impact the accuracy of our customer's financial statements.  Our customers have expressed the need to obtain assurance pertaining to our internal controls as they relate to the security, integrity, confidentiality, and privacy of the data we process, store, or transmit for them, as well as the availability of the systems we host for them.  Consequently, we have selected to undergo a SOC 2 engagement that is based on the same family of standards as SSAE 16, but is more applicable to the services we provide our customers.  

Our SOC 2 report, based on SSAE 10, 11, 12, and 14; cumulatively referred to as AT-101, provides assurance regarding security, availability, processing integrity, confidentiality, and privacy.  It is not intended to provide our customers with assurance relevant to internal controls over financial reporting (ICFR).  If your internal or external auditors require supporting documentation regarding how we determined that our services  do not pertain to our customer's ICFR process, we will be happy to provide the information to them upon request.

Thank you very much for your business.

You will have to tailor the references to all five TSPC principles if you only received an attestation on a subset of the five (ie. just security and availability).

If your CPA firm has not helped you document why your internal control environment does not have ICFR impact on your customers, then you can contact me at jlong@compliancepoint.com, and I will document it for you using the Degrees of Risk Separation (DoRS) approach to determining ICFR.

Your Vendor's SOC 2 Report may be Inadequate

It took a year and a half, but the market finally understood that SOC 1 (aka SSAE 16) cannot be relied upon for assurance regarding security, availability, processing integrity, confidentiality, or privacy (SAPICP).  The misuses of SOC 1 that I chronicled since December of 2011 have started to taper off, and people are asking for SOC 2, the correct report for assurance regarding SAPICP.  Great.  

Now the challenge for everyone receiving SOC 2 reports from their vendors is the question of adequacy.  For example, take a look at your vendor's controls associated with Security Principle 1.2d "Preventing Unauthorized Access."  This control is a good indicator of whether the company had a thorough assessment of their internal controls or not.

One company's failure to prevent unauthorized access was what enabled a U.S. developer to outsource his own job to China.  The point of failure in this case was the company's lack of VPN concentrator log reviews.  Does your vendor's internal control associated with Security 1.2d or 4.1 include a daily review of VPN connections to the network by a competent and independent person?  What are the criteria that would trigger an investigation?  Keep in mind that internal threats are just as important to consider as external ones.

It is a good idea to compare your own organization's security controls with those listed in your vendor's SOC 2.  If your vendor's SOC 2 does not measure up, their SOC 2 report is inadequate.  Consider sending your security questionnaire to your vendor, and asking them to explain how each control in their SOC 2 maps to your questionnaire.  Below are some control areas that should be addressed in Security 1.2d 

• Remote access controls (including monitoring of logs)
• System account creation, modification, or deletion
• Root admin accounts usage (Unix/Linux environments)
• Service account controls
• Network equipment access controls
• Domain administrator access controls
• Inactivity timer/ automatic log-off
• Account lockout 
• Account re-certification 
• Disabling accounts inactive more than 90 days 
• Sharing of user accounts and passwords
• First time passwords 
• Password history, aging, and length
• Requests for user password reset 
• Password exceptions – non-expiring passwords
• Storing passwords in readable form
• Password complexity

Even within the control areas above, there is opportunity for weak security controls.  One of my favorite vulnerabilities to point out is the minimum password age setting.  For example, if you require password changes every 90 days, and have a 5 passwords remembered policy, but have a minimum password age set to 0, then I can simply change my password 6 times every 90 days, and get back to the password I like.  There are a million examples like this where seemingly secure control environments can be bypassed.

If you are a service organization seeking to provide your customers with a SOC 2 report that will not be rejected as inadequate, I recommend choosing a CPA firm that has security as one their core competencies, or one who has partnered with a security firm like CompliancePoint.