Guest contributors
Hot topics
Business cases
Experts voice
At a glance
PDF version

Subscribe to
Bull Direct:


n°20  |  November   2007
Experts voice

Information systems security audits – the current state of play
By Lionel Mourer, Director of the IT security Consulting and Audit business, Networks and Security Division, Bull Services France

Lionel Mourer has more than ten years experience of strategic and operational consulting in IT security for major companies and a large number of small-to-medium businesses. He joined Bull in 2004 to set up and grow the IS security Consulting and Audit business. In addition to his role extending the offering and managing his 15-strong team, Lionel is involved into numerous pre-sales projects, and also contributes to the production phases on a number of consultancy assignments, and helps manage complex projects. Lionel qualified at the French telecoms engineering school, Télécom Lille 1, as an ICT engineer.


The question is as old as the notion of IS Security itself: which is preferable, an organizational audit or a technical one? And moreover, what are we really talking about? Two kinds of audit that, at the end of the day, complement, rather than compete against each other…

There are two families of IS security audit: technical and organizational. Let’s start by examining the content of these different audits.

The organizational audit, focused around risk analysis methods
Risk analysis using methods such as MEHARI® (developed by CLUSIF, the Information Security Club of France) or EBIOS® (from the French government’s central IS security department, the DCSSI) enables us to identify and implement appropriate measures for securing the information system in line with the business challenges being faced. This serves as the basis for drawing up an IS Security Policy and Security Master Plan. The procedure (see figure) highlights the current IT security status in relation to the enterprise’s key concerns and the risks the company wants to consider (those not included are referred to as ‘residual risks’): these have to be identifiable when the information system evolves. This forms the basis on which generic recommendations can be made for the enterprise to implement.

Fig 1

Fig 1. Generic procedure for a risk analysis method (source: Bull)

Nonetheless, at this stage risk analysis does not enable us to prioritize our recommendations. For the organizational audit to be complete, a classification of the recommendations aligning them with the enterprise’s concerns and resources has to be conducted. Depending on the enterprise’s needs, the organizational audit may be applied to one particular area of the IS – for example, an application – or to the whole of the information system.

It is still true that an organizational audit is often perceived to be rather ‘heavy going’, complicated to implement, very time consuming, and so relatively expensive. But while formally applying a risk analysis method can easily take 100 days or more, it remains possible, with good experience of tools and organizations, to carry out organizational audits in less than 20 days: this can be achieved by only selecting a subset adapted to the company’s business activity, while remaining relevant with regards to the key challenges.

Technical audits vary according to need
The aim of these audits is to ensure that both technologies and people who have implemented them are mature, by analyzing the impact both have on IT security, and by determining for each of them the level of availability, integrity, and confidentiality implemented as compared with a benchmark for the enterprise.

There are different types of technical audit, including:

  • Architecture audits: here the aim is to diagnose the security situation for a platform or solution: to define weaknesses and strong points in the links making up the application chain, prepare an inventory and qualify existing vulnerabilities, and identify the security countermeasures to remedy them
  • Vulnerability tests: these enable an inventory of faults/weak points on one or several systems (server, router, etc.) to be created. They can be run internally or externally, on a single target, or anything up to several thousand targets
  • Intrusion tests: these aim to qualify the level of IS security from outside or inside the enterprise. These procedures are based on a rigorous methodology enabling constant service quality to be guaranteed, ensuring that over-loading does not occur, and providing a system for event traceability, etc.
  • Audits applied to code: ‘re-reading’ the source code for an application from the security point of view can help identify programming techniques that do not provide a sufficiently high level of security, or the presence of vulnerabilities likely to lead to security problems
  • Configuration analyses: these enable local vulnerabilities within a basic system component to be identified. A router, for example, would correspond to a single basic component, while a Web server is a functional component made up of three elementary components: the operating system (e.g. Linux), the Web application (Apache, for example) and the specific development work carried out by the customer. The critical parameters for the configuration of each elementary component are validated with reference to the client’s existing benchmarks and Bull’s own security guidelines.

Technical vs organizational implementation?
Generally speaking, the organizational audit enables us to acquire sufficient amounts of company-critical data and information about the risk the enterprise is subject to, whereas a technical audit is aimed more at a customer that is already mature in security terms, and so the requirement could be, for example, to simply validate one or several components within the information system.

This naturally places the organizational audit as just the first step in a complete process, which, typically, will lead eventually to one or more technical audits. In effect, once the organizational audit has been carried out, an action plan consisting of various IT security projects needs to be implemented, including architecture audits, vulnerability tests, intrusion tests, etc. In fact, an organizational audit is in no way a substitute for a technical audit, and vice versa.

On the other hand, enterprise IS are constantly evolving, and business concerns also change. What’s more, threats are on the increase (for example, over 100 new bugs appear every week1 ) and risk themselves also change (who would have thought, for example, before 11 September 2001, that a skyscraper could just disappear following the impact of a single aircraft?). In terms of IT security, more than anywhere else: “If you’re not moving forward, you are going backwards!”

So today’s enterprise is faced with a fast-moving race: hence the need for regular security audits in order to take these new challenges, vulnerabilities and risks into account. Recurrence is, by definition, variable:

  • The organizational audit has to be repeated, either systematically every year to bring the enterprise’s security policy up to date, or when the enterprise implements a new strategic application, a new IT architecture, etc.; in other words, for each major evolution of its IS
  • As for the technical audit, this can be:
    • Frequently recurring (this is true for vulnerability tests applied in ASP mode, or via an appliance connected permanently to the customer’s IS)
    • Carried out during the production phase of a new infrastructure or application (for example, via an intrusion test, or configuration analyses).

It remains for the security audit to be to be positioned within the organization’s structure

  • Increasingly, audits are commissioned by the Chief Security Officer, but this could equally be the Chief Information Officer, an operations manager, a management control and/or internal audit department, etc. We also see audits being commissioned by external entities (customers, certification companies, insurance companies, etc)
  • In the case of the organizational audit (in particular), it is always relevant that senior management are involved. In effect, when undertaking a risk analysis operation, it is always a good idea to meet (ideally) all the functional Directors in the organization: this means support from the Chief Executive or Managing Director is often required
  • On the technical audit side, some procedures (intrusion tests, for example) require approval from an employee representative (or equivalent) for the enterprise.

Audit results must be presented to staff in the enterprise on a ‘need-to-know’ basis. Here again, these people will vary depending on the type of audit and what it is setting out to achieve: from the system administrator to the General Manager, and including the Chief Security Officer, etc. Finally, these results could include a security control panel, or even lead to changes – for those enterprises that are most advanced in terms of IT Security – in the IT security management system.

Some of our success stories…
Here are a few examples of typical scenarios undertaken each year:

  • A recently appointed Chief Security Officer for a particular organization wanted to run an awareness campaign for senior management – the latter lacking a broad vision in the face of IS security problems, and allocating insufficient funds to this area. Bull suggested they run an ‘awareness raising’ intrusion test. In this scenario, the Chief Security Officer expected the test to be able to demonstrate the potential capacity of external intruder to access the organization’s IS. In principle, the ‘awareness-raising’ intrusion test leaves a wide field of action open for the intruder to gain entry (via all possible ‘doors’ to the system). The results of the test, once they had been presented to the GM, led to the security department being given adequate funding to launch a much wider security project, based on an organizational audit, and taking into account all the enterprise’s IS.

  • This customer manages a network outsourcing contract for a major international bank. The contract set out to validate the level of network security by comparing it to the bank’s internal IT security benchmarks, confirm the discrepancies identified, and to do this once every year through a third party. Bull suggested carrying out:
    • External tests via the Internet, RAS, a supplier’s LAN, and via the LAN of another customer
    • Internal tests (on 11 servers)
    • Telephone tests (using a range of 500 numbers, as well as intrusion tests)
    • Configuration analyses (nine servers, 15 network components and one mobile workstation).
    The results of the audits meant that some vulnerabilities were corrected, but also that the bank’s internal standards were enhanced.

  • Following a major problem with its IS, another customer was having great difficulty restarting its system, due to the fact that back-ups had been created randomly. Bull carried out a security analysis for a complex series of back-ups in three stages:

    1. Stage 1: exhaustive study of the current state of affairs
    o Organizational analysis: provision of a detailed questionnaire on back-up procedures, selecting the most relevant aspects of several methods or IT security ‘best practices’ (Méhari, EBIOS, ISO 17799:2005, etc.)
    o Functional and technical analyses of the back-up mechanisms that had been implemented
    o Interviewing different users of the back-up system (both organizational and technical)

    2. Stage 2: analysis of current and future needs.
    This phase enabled us to compare the reality on the ground, evaluated in Stage 1, with the actual back-up management requirement. It included:
    o Analysis report on the main issues and challenges
    o Analysis of need in terms of volume and the nature of the information being handled (e.g., how critical it is)
    o Analysis of performance requirements
    o Organic analysis.

    3. Stage 3: recommendations:
    o Organizational report (teams, skills, processes, documentation)
    o Possible improvements on the current solution
    o Suggested back-up strategy for solving the customer’s issues.
    These recommendations were prioritized (in relation to the issues and budgets involved) and supplied in the form of an action plan to be undertaken by the customer. The results enabled a back-up/disaster recovery policy for the enterprise’s entire IS to be implemented.

1 Cf. article in Bull Direct n°6, July 2006 “ The challenges of patch management: managing corrections, alternatives and best practice, day to day ”.

More information:


I know just how to go about finding a file on an inadequately protected server,” comments Jean-Christophe Touvet, Senior Consultant and Manager responsible for Bull’s Technical Audits offering. “But I have no idea what value that file really has! The issue is therefore to work out whether the file has any significance, and this is the role of an organizational audit. A technical audit only serves to prove that such a file is accessible.
For it to be as relevant as possible, an audit can only be designed on an organizational level to begin with, before it can come in the technical details. But this approach does not seem to have gained widespread acceptance as yet. “When a customer is not mature in IT security terms, we will often be asked just to get involved with the components of the system, rather than analyzing the whole IS. But we need to analyze the enterprise’s business activity and requirements to discover the critical issues involved. Without this, the consultant’s role is reduced to simply recommending products,” explains Lionel Mourer.

Contact  |  Site map  |  Legal  |  Privacy