What is a Disaster Recovery Plan?

Disaster recovery planning is an IT function often involving a whole of business team whose role it is to anticipate disasters of any scale, determine the effects these would have on business continuity, and then create a set of policies and procedures for minimizing downtime and expediting recovery to pre-disaster levels.

In smaller corporations the disaster recovery plan may in fact be just a sheets of paper listing steps to take when the unforeseen occurs, in fact, smaller organizations may be able to download disaster recovery templates from expert websites and simply follow the instructions for completing the audit and creating the recovery plan.

However, larger corporations and the government sector typically require a great deal more detail in their disaster recovery plan which will more likely be a set of procedures and policies filling several folders and requiring extensive staff training from board level right down to data entry operators.

Planning for disaster recovery requires detailed business risk analysis, and a keen understanding of the effect various disasters will likely have on the business, from temporary setbacks that can be coped with, right thru to disasters that threaten the viability of the corporation in the short and long term after the plan is implemented.

What is a Disaster Recovery Plan? Disaster Recovery Plan

In the pre-planning stage an audit by certified disaster recovery experts should be carried out. This may take considerable time to complete but is absolutely necessary for ensuring some critical measure is not forgotten. A small detail could have huge ramifications.

Probably the most important step in preparing a disaster recovery plan, the pre-planning stage will pick up potential threats to data storage, specifically whether off-site backups will be affected by the same disaster that incapacitates the live data. Similarly, a professionally undertaken audit should be able to establish if redundancies in networking will be affected by the disaster.

Having determined the threats to the business data, analysis of recovery options and budgeting for them takes place, and at this stage the complete plan takes shape. This is a commercially sensitive document and is usually known in its entirety to only the most trusted corporate officers.

No disaster recovery plan is ever complete without also testing its efficacy, a process that allows for fine tuning and fault analysis well before ever having to rely on the plan for business continuity. Testing the recovery plan should be considered a necessary expense of producing the plan rather than an unneeded cost after the fact. Far too many corporations neglect testing their systems and procedures, resulting in business closure.

Once the disaster recovery plan is complete, any authorized officer of the organization should be able to refer to it in the event of a major disaster, and sad to say, this means not keeping the plan in soft copy on the Intranet. Printed copies need to be distributed to relevant personnel, and better yet, a copy of the plan needs to be stored off-site. Never take for granted the security of the head office.

What is Disaster Recovery?

Disasters happen, be they manmade or natural, no corporation or government can ever assume their data and access to their data will be exempt from damage. Disaster recovery is their ability to recover electronic data and data processing to pre-disaster levels in the shortest possible time thru forward planning and setting policy objectives.

Preparing for disasters and knowing that recovery will be possible requires careful analysis of the threats facing a corporation such as earthquakes, terrorism, hacking, staff strikes, electrical failure, even human error, and understanding the consequences for data of each possible situation.

What is Disaster Recovery? disaster recovery

The simple activity of keeping off-site data backups may be all that is required for recovery in smaller corporations and private households, but in larger corporations most responsible IT managers would consider this a basic minimum which would already be provisioned and costed as part of their annual budgets.

Some disasters on the other hand have very little to do with recovering backups, a denial of service attack (DoS) on the corporate website could happen at any time, and all the backups in the world won’t help the corporation with their recovery, instead skilled manpower is needed. This is the sort of disaster that affects a corporation’s ability to use it’s data and communicate with stakeholders, and if not contained quickly affects credibility and stockholder confidence.

Forward planning for eventualities such as DoS attacks and other forms of hacking or viruses often involves contracting third parties for network routing and emergency response. Common network related disasters include hacking, viruses, server and router failure, cable fractures or satellite failure, or any number of other failures in the network loop.

Protecting a corporation from disasters is costly, and often considered a wasted expense until after a major disaster has occurred, and business recovery specialists often complain of the difficulties they face convincing management and staff of the need to work to set procedures.

Equipment and network redundancy plays a large role in mission critical disaster recovery, and may in extreme situations require two or three completely separate systems performing the same task. Many Fortune 500 corporations may not need completely redundant systems that can be brought online almost instantly although this is common in larger financial corporations, and of course large telecommunications operators.

The construction of disaster proof data centers with high levels of redundancy, fire containment, and temporary power generation is often seen as integral to disaster recovery planning, as well as provision of redundant network loops to third parties. Most major corporations and governments allocate as much as 5% of their annual IT budget to disaster recovery planning, a high expense, yet preferable to not continuing in business in the event of catastrophic data loss.

Preparing for every possible contingency would require superhuman effort and unlimited funds however using the Pareto Principle (80-20 rule) and planning in advance for the few disasters that could permanently shut down the corporation is much more affordable, and once these steps are in place the incremental cost of adding more preparedness may not cost that much more.

What Are Disaster Recovery Services

Disaster recovery services are those usually provided by third party contractors in the event of  disaster affecting IT operations in-house. Most providers offer dedicated data centers and networking capabilities to ensure survival of data and continuity of business in the days and weeks after allowing time for full recovery or relocation of the organization’s IT infrastructure.

Being prepared for unforeseen disasters requires extensive pre-disaster planning and consultation with major stakeholders as well as service providers to ensure business operations are protected, namely that data and configuration backups are stored securely, and will be available very short term or in real time when needed.

Data though is just one aspect of disaster recovery, the others being communications and facilities. Since September 11th many corporations are now required by their stockholders and by government regulation to ensure continuity of business operations if one or several branch offices are destroyed, even to the extent of relocating key staff or splitting functions across diverse locations.

Service providers make all of this easy by investing in staff and assets and focusing on their core business of helping corporations recover from disasters. By contracting with third party service providers the IT administrator is able to focus on operations, the job of monitoring and reporting on backup systems being effectively delegated.

Major vendors of disaster recovery services all offer hot site failover. with the result the end user who may be located on the other side of the world will not even be aware the primary data center has been affected. Customers and other stakeholders of the corporation will still expect reliable service, even if the corporate head office has been destroyed.

What Are Disaster Recovery Services disaster recovery services

Disaster recovery data centers provided by service providers allow for complete duplicates of the corporation’s IT infrastructure to be created and mirror all processes ready to take over in an instant, and whilst this is an expensive precaution its value is negligible for corporations that cost seconds of downtime in the millions of dollars.

Offsite storage of hardware that can be brought online in the event of a disaster is a more affordable solution for smaller corporations that only requires specifying the hardware and software needed and making sure that service providers are able to restore data to the backup servers on activation.

Many corporations especially at the smaller end of the scale without a highly skilled team of IT professionals working in-house, often neglect to plan for continuity of communications such as email.

Disaster recovery services that offer routing of email communication to third party servers are easily configured using DNS, prevent loss of email and enable staff to respond as soon as service is restored.

An increasingly popular service offered by major vendors is work area recovery in the event of the corporate offices becoming unusable. Vendors undertake to make office space available in their own premises with pre-connected workstations and telecommunications equipment, and all of the networking requirements provided. In extreme situations portable offices housed within containers can be made available.

How to Get Disaster Recovery Certification

Disaster recovery certification for IT professionals is becoming a necessary pre-requisite for anyone involved in business continuity planning, which has taken on added importance since the unfortunate events of September 11th in New York. In the past, IT professionals would plan for major systems faults, but now planning for entire data center destruction is necessary.

Business continuity is taken seriously by management in most major corporations, and IT staff with disaster recovery certification are in demand, so much so, that salary and benefits are presently higher for qualified experts. For the foreseeable future, certification from professional IT institutes are likely to better received than general information systems diplomas, mostly due to the complexity of the subject and importance of disaster recovery.

Current certification programs are for existing IT personnel already employed in the role and require a minimum amount of experience before being eligible for training, although there does seem to be some flexibility in this rule.

Studying online is a popular option for many hard working IT professionals who are able use quiet time in the office to work thru course notes and exercises. Finding a reputable online training provider offering disaster recovery certification is important, every certification provider is different and not all courses are created equal.

How to Get Disaster Recovery Certification Disaster Recovery Certificate

Consideration should also be given to the assessment criteria for disaster recovery certification  examinations, will candidates be able to complete this within the workplace or locally, or is extensive travel to another city required, and if so, how much time away from home needs to be allocated.

Conversely, on campus study may be a better option for many students with two or four day seminar style course available. Undertaking disaster recovery certification in person exposes the student to other IT professionals also working towards certification giving opportunities for networking which may be invaluable as disaster recovery planning is undertaken within the workplace.

Examinations for disaster recovery certification need to be prepared for since a minimum 75% pass is required to graduate from many courses. The cost of taking the exam is levied for each separate time the exam is taken, and with prices up to several hundred dollars failing and needing to resit could be costly, and is an expense most employers would be unhappy to pay twice.

Re-certification every year or every other year may be a constituent requirement of the disaster recovery certification course undertaken, particularly for professionals working in senior business continuity roles within IT. Reasons for ongoing assessment are related to the speed of technology change, something most IT personnel are used to, but with advances in server and networking systems occurring every few months, ongoing assessment is designed to ensure graduates are always working with current theory.

Some of the most popular qualifications for IT professionals in disaster recovery include the Associate Business Continuity Professional (ABCP),  Certified Business Continuity Professional (CBCP), Master Business Continuity Professional (MBCP), Business Continuity Certified Planner (BCCP), Business Continuity Certified Specialist (BCCS), Disaster Recovery Certified Specialist (DRCS), Business Continuity Certified Expert (BCCE), Disaster Recovery Certified Expert (DRCE), and Master of Science in Information Security (MSISE).

Disaster Recovery Solutions

When disaster strikes your IT infrastructure or network, having quick access to disaster recovery solutions is paramount if the system to is to be restored to full service. Whether they are in-house or outsourced, the solutions called for in the disaster recovery plan need to be available in a timely and reliable fashion.

Disasters are by their nature unpredictable, difficult to anticipate, and usually very inconvenient to users of the system. Little can be done to prevent many disasters, especially natural, but even man made disasters are often not easily prevented, hacking attacks or denial of service attacks are rarely advertised before the event.

Proper planning for disaster recovery solutions is therefore one of the primary roles of senior IT staff, who are required to analyze all systems and threats in detail, finding specific failure points, before beginning the process of setting goals and determining possible solutions in the event of disasters.

In smaller corporations it is often better to outsource disaster recovery solutions to the contractor responsible for building and maintaining the network, who are better placed to understand the threats faced and make contingency plans for retrieving off-site backups or swapping out problem components.

Disaster Recovery Solutions recovery solutions

Outsourced disaster recovery solutions are often considered an efficient use of company resources allowing the IT team to focus their energies on improving and maintaining existing systems. Cost wise, outsourced solutions mean that backup components don’t require capital investment, but are still available when required.

By contrast, internally sourced backup and redundant components offer peace of mind and security in larger corporations where specific components are chosen for compatibility and reliability, and where unknown components are considered a risk to the overall goals of providing uninterrupted service levels.

Disaster recovery can take many forms, each solution potentially requiring different resources and implementation. Similarly, scaled solutions often require vastly different implementations, for example the need for off-site backups may not require much more than contracting a security firm to collect daily backups, or it may require real-time backup synchronization on redundant systems that are capable of taking over instantly.

Successfully recovering from disaster is not luck, very few corporations have remained viable businesses after a disaster has struck, the few who do service have demonstrated a willingness to invest in solutions, preparing for the unforeseen, and have plans and systems in place to cope. Larger corporations are at the same time more vulnerable, and yet also more resilient, than small localized businesses, but once the market has lost faith in their ability to recover from disaster, their days are numbered.

Those corporations that seem to weather the storm more successfully than others also tend to have IT managers who work well with other senior management and adopt a whole-of-business outlook. The reality is that any disaster recovery solutions planned and then implemented need the support of key personnel in other departments if fiasco is to be avoided.

Staff training, and the insistence of senior management that policy is followed are critical to the success of any disaster recovery. Solutions need to be implemented across multiple departments and often disparate locations. The IT department needs to know that it can carry out it’s function without undue influence from nervous departmental managers, and that the solutions they implement are accepted.

Disaster Recovery Software Options

Catastrophic data loss is any IT administrator’s worst nightmare and disasters can strike at any time. Evaluating the available disaster recovery software options as part of the organizational disaster planning is often overlooked in favor of adopting known packages that may not in fact be the best choice.

Disaster recovery software covers a range of niches, from backup and recovery, to mirroring, network monitoring, data salvaging from corrupt disks and more, so there certainly are enough options available for protecting the corporation’s critical data.

Software that allows a hard drive to be imaged and then cloned across multiple systems or servers has taken on added importance since the advent of the disaster recovery industry and is no longer restricted to being used solely for rapid deployment, although this is certainly its main function. In disaster planning new servers can be kept in storage and cloned relatively easily.

Disaster Recovery Software Options recovery software options

Disasters affecting servers or buildings often tend to destroy more than just data, so imaging an identical system and then running a data restore reduces downtime considerably. If the corporation cope can with the downtime required to clone and restore a server this is a valuable option for the IT administrator.

A significant problem in many smaller corporations is the mindset that backing up the company accounts and documents is sufficient, meaning that system and network configuration files, user settings including calendar and browser passwords, mail server data, even web server data are lost. Sadly this is why many smaller corporations fail after a disaster.

Backup software are not all created equal and relying on the backup scheduling inherent with most operating systems is just not good enough. Additional software is almost always recommended that is designed specifically for disaster situations as well as archival purposes. Backup solutions that scale with the growth of the network should be considered essential.

File mirroring software has in recent years matured into a solid and dependable technology giving IT administrators an extra option in their arsenal of disaster recovery software. Mirroring software eliminates the need for identical hard drives or dedicated server links, the only requirement being similar or greater storage capacity and reliable network uptime.

Software mirroring is generally considered more flexible allowing IT to configure specific settings not available at I/O level, or indeed over wide area networks using VPN tunneling. In addition, software mirroring may not require identical operating systems to be installed on the source and target servers.

Data loss from a failed hard drive is minimized thru backups and mirroring, yet even then it may still be necessary to recover data from a failed hard drive, particularly when users don’t follow policy and save files to their local drive. Ignoring this disaster may not be an option if the CEO is the guilty user and insists on software recovery.

Not looking into disaster recovery software options thoroughly enough at the disaster recovery planning stage might affect business continuity after the disaster, and would be a career breaker for any IT administrator.

Disaster Recovery Best Practices

Disasters occur, and very rarely according to anticipated schedules. Given the overwhelming reliance on IT in most organizations a disaster has the potential to force out of business the unprepared making disaster recovery and adherence to best practices vitally important. Any organization without a disaster recovery plan is courting fate and is unlikely to survive.

Disaster recovery in an IT context is not the same as high availability and the two should not be confused, yet in many organizations this is the case. High availability defines a network that is online most of the time, yet even the best planning cannot completely eliminate downtime caused by disaster such as staff being unable to enter the workplace and implement the disaster recovery plan.

Yet despite the obvious differences, following best practices is capable of narrowing the gap and in less serious disasters it may be possible for a network to remain operational even if large parts of it  are incapacitated or destroyed. Actively seeking the advice of disaster recovery experts and vendors at the planning stage brings relevant expertise into play.

Disaster Recovery Best Practices Database Recovery

Ensure the plan can be implemented by staff outside of the planning process, the authors may be unavailable in a disaster or may have left the organization so clearly written policies and procedures are desirable outcomes. Using plain English understood outside of the IT world, and providing training to all staff on disaster preparedness is likely to offer more opportunities of successful recovery.

An audit of the disaster recovery plan at regular intervals, perhaps every year, and empowering staff to actively engage the audit looking for improvements will allow better long term planning decisions to be made. The IT administrator or CIO and business continuity team need to be involved in setting the scope of the audit. At senior management level the audit report needs to be to discussed and understood in terms of contingencies and future strategic planning.

Best practices in drafting the disaster recovery plan will dictate the goals are clearly defined and written down in detail with the aim of full technical recovery despite the worst disaster that could occur. Disasters upto and including pandemics, terrorism attacks, the threat of war, and natural disasters can all be planned against.

Depending on the scope of the disaster significant parts of the business may be destroyed, but good communication of the recovery goals and procedures should be sufficient to ensure survival of the business. Documenting data restoration procedures along with visual media such as screenshots or video presentations, along with locations of supplies and media satisfies common best practice guidelines.

A full test of the disaster recovery plan, and regular testing of isolated parts of the plan highlights any issues that could arise during an actual disaster. Simulations should not be relied on to identify potential problems, as it might not uncover issues of comprehension amongst non-IT staff or  potential resourcing issues from senior staff not being available as might be the case if passwords, PIN numbers, or keys are required as part of the restoration.

Disaster Recovery and Business Continuity Auditing

Between forty and sixty percent of all businesses that suffer a business threatening disaster fail in the next five years so disaster planning is not taken lightly by stockholders, or regulatory authorities. Conducting an audit of the disaster recovery plan will immediately make obvious any discrepancy that would affect business continuity

Disasters occur all the time, and are rarely anticipated. Many IT administrators and organizations plan for hardware failure in their networks or attacks from hackers and viruses, and understandably many will have no plans for natural disasters such as earthquakes, flooding or fire. Even fewer will have plans in place to cope with terrorism or pandemics such as the much anticipated bird flu.

Business continuity planning is a vital task of all departments even though it might be coordinated by IT or primarily involve disaster recovery of data. Of course all departments need to take responsibility for their specific roles under the disaster recovery plan. The written plan in a large organization may extend to several hundred pages, and would be difficult to keep updated without frequent testing of procedures and regular audits.

No audit of disaster recovery procedures would be complete without first reviewing the plan and its documentation. Staff and third party contact details need to be up to date, and all staff with responsibilities under the plan issued with detailed instructions. Training of staff that is adequate for the duties need to be arranged and tested, and new staff inducted into the company disaster recovery procedures.

Disaster Recovery and Business Continuity Auditing Business Continuity Auditing

Company insurance policies need to maintained at levels suitable for expected losses after a disaster, the audit should establish that these are paid for or if paid monthly or quarterly are not in arrears. Similarly, the audit will be investigating third party contractors and ensuring all networking, backup, mirroring, and data center provisioning is in place as per agreed deliverables.

Backup procedures are an important component of disaster recovery and should be tested for integrity and completeness on a regular schedule with occasional simulation of a failed server. Auditing teams will be looking at the efficiency of restoring procedures and whether full service is resumed, as well as systems in place to catch incomplete data restoration.

If fail over networks, redundant servers and spare critical components are factored into the business continuity plan then auditing their usefulness and response times will be considered. Off-site locations will be evaluated for suitability, disaster immunity, and security of data. If a hot site is provisioned, auditors will want to eliminate deficiencies that may affect the success of disaster recovery

Performance of disaster recovery plan duties by staff and third party contractors at audit time are not left untested, business continuity depends on the people implementing the plan to get it right. Excellent communication skills and expertise in their role can be evaluated against industry standards in addition to running simulations with staff.

The auditors job is to ensure that disaster preparedness is as complete as humanly possible, so staff shouldn’t feel threatened. During a disaster recovery and business continuity audit, management need to allay the fears of staff, yet also reinforce that ultimately the corporation cannot afford to not be prepared.

Where to Get Disaster Recovery Training

Disasters sadly are a fact of life, most of us manage to avoid them although this is probably more to do with circumstance than any activity on our part. In IT terms the same applies, yet with the benefit of disaster recovery training the effects can be minimized, although never completely eliminated. The probability of disaster striking at some point thru no fault of the IT manager and her team is high, be it hacker attack, or earthquake, the end result is always still the same.

Preparing for disasters and being able to recover from them requires scientific thinking and analysis of the threats facing the corporation, and without training in disaster recovery is difficult to fully anticipate. Disaster planning is not an activity for informal groups who brainstorm and put together quick procedures.

Instead, disaster planning is a serious endeavor that is literally going to make the difference between business continuity or closure, or just maybe a hostile takeover, either way, no IT manager or senior manager wants to be responsible for the corporation failing due to a disaster that could have been mitigated.

Staff training for disaster recovery is best undertaken thru a professional organization whose methodology is acknowledged as best of class, and just as importantly, whose trainers have real world experience of working in an environment where they were responsible for disaster planning and then had disaster strike, thus validating their skills.

Where to Get Disaster Recovery Training Disastar Recovery

Finding a training provider in your area may not be possible, after all, disaster planning in an IT role  is likely going to require very different knowledge than would be appropriate for general management. Local IT networking clubs may be sufficiently large enough to have a training arm, at least in larger cities this could be expected to be the case, yet even if they don’t, they may still possess a library with industry recognized handbooks on disaster recovery.

A number of training providers operate online using websites with interactive modules and can be an excellent resource for IT staff who lack the time to leave the office for extended periods of time. Online courses also tend to be more affordable although it is important to be sure that assignments and tests are marked by humans with relevant experience.

State colleges and universities with their own information technology department will also be expected to offer disaster recovery training, and often have the added advantage of laboratories where course exercises can be tested without potential harm to the corporate systems. Further advantages include access to library resources and studying with other IT professionals from outside the workplace.

Networking vendors and server manufacturers may offer the best training, provided their course is broad enough to be useful outside of their own product specialization. Typically, full service vendors would be more capable, and probably too their training department which would have staff with experience of current real world scenarios that affect entire systems or networks. Certification from a vendor can be a very rewarding qualification to hold, useful far beyond the disaster recovery plan.