cloud-network_fklxrthd-1024x768

Cloud Integration: Patterns, Strategy, Architecture

According to a recent survey by the Right Scale blog, of a total 1002 IT professionals interviewed, 95% of respondents confirmed that they had already adopted some form of cloud service integration.  Interestingly, 67% of those are using a combined public and private cloud hybrid solution, proving that the majority of businesses believe that multi-layered cloud solutions are the most effective way to improve their business practices.

For me, these figures are definitive proof that you risk harming your business if you don’t hurry up and initiate a cloud integration strategy very soon. Understandably, many companies are reluctant to make the switch for a number of reasons; cost, not wishing rock the boat, concerns over the need for staff retraining, and a belief that the cloud cannot offer them the specialized programs and tools that their business requires etc.

I feel that this viewpoint is exacerbated by the unclear and inaccurate information that is swirling around the internet. Having spent a number of days trawling through numerous articles on the subject, I must be honest to say that I did find my jaw slightly aghast at some of the advice and information that I came across in regards to both choosing the best cloud platform and how to then go about integrating it into your existing architecture.

In this article, I aim to give an overview of cloud integration, both in terms of its architecture and the best strategies for implementation. I intend to begin with a brief explanation as to why choosing the correct method for cloud integration is so important, before then going into cover the different types of cloud architectures in more depth.

Contents

Why businesses fear the cloud
How to move forward with cloud integration
Main types of cloud platforms:
Case Study – SaaS integration
The main challenges of SaaS integration
What are the main issues companies will face during SaaS integration?
Cloud integration: Strategy for implementation
The two main types of SaaS integration:
Using Cloud Integration Patterns to overcome integration problems
Final Thoughts

Why businesses fear the cloud

having-computer-problem

Let’s say that you are a medium-sized company that is doing pretty well. A few years ago you upgraded your entire computer system at considerable cost. The upgrade caused quite a disturbance to your company, during office hours there were lots of new people around the place getting in the way, workstations started to disappear meaning staff had to be shuffled around the office until they were replaced, staff had to be taken off their duties to be retrained on the new system, and worst of all, parts of the new system didn’t work at first, kept going offline, or didn’t have all the data from the old system that was needed. You remember that it took several months to iron out all the problems but today, despite having a few more grey hairs on your head, you recognize that the upgrade was necessary as the old system was outdated and had to be upgraded.

blog-banner-download-project-specs-2

Recently, you have read about your competitors who have just migrated many of their systems/applications to the cloud and are now singing its praises. You begin to do some research and find that the cloud can do all the same things your existing system can while at the same time offering enhanced security and several other potential benefits, many of which you realize might help you streamline your business significantly in the future.

You think over your past business strategy and quickly come to the conclusion that it was by keeping up with technology that was a big part of the reason your company was able to grow so successfully. Everything from digitizing your filing system to your web-page has helped boost your company. For this reason, you begin to consider cloud implementation more seriously. After reading a plethora of articles that fully support cloud integration, you come across this set of statistics published by Oracle as part of their publication on 5 Ways to Simplify Cloud Integration:

cloud-statistics

Suddenly, all those fears and apprehensions come flooding back. What should you do?

“The supreme art of war is to subdue the enemy without fighting” – Sun Tzu

The most important thing is to keep sight of your goals. Whether or not it is the right moment to integrate cloud applications and services into your existing system is a matter for you to decide. A good idea is to do a detailed review of present and anticipated future needs to ascertain if it is the right time. The reality is that at some point your company will need to move its systems to the cloud as this is the direction that things are inevitably heading.

I strongly recommend that you take the time to do a review, which if the relevant expertise is not available in-house, should include some level of consultation with several professional cloud integration services.


How to move forward with cloud integration:

There are 3 options when it comes to how to undertake cloud integration:

  1. Outsource 100% of the job to a professional cloud integration service
  2. Partial use of outsourced professionals alongside in-house IT team
  3. In-house cloud integration

The obvious difference between these 3 options is cost. While any business would love to have the resources on hand to pay a company to come in and take care of all aspects of the cloud migration, the fact is that most simply can’t afford to do so.  Which option companies take really comes down to budget and the number of in-house IT professionals on hand. Many companies choose to go it alone and rely on the expertise of the cloud platforms themselves when problems arise. Most platforms have well-trained customer service teams that can be contacted through live chat functions, email or by phone.

Main types of cloud platforms:

pas_ias_sas

IaaS – Infrastructure-as-a-service

SaaS – Software-as-a-service

PaaS – Platform-as-a-service

Case Study – SaaS integration

cloud-integration-patterns-for-it-pros-itprceed-4-638

Though there have been lots of interesting developments in regards to iPaaS integration, particularly since the advent of the hybrid IaaS model, unfortunately, I only have time enough to focus on the hybrid SaaS model of cloud integration. The reason for this choice is primarily because SaaS is the type of integration that causes the most problems and still harbors the greatest challenges.

blog-banner-download-project-specs-2

The SaaS model was a game changer so far as most businesses were concerned. It facilitated the use of powerful applications and services, all hosted on the cloud, by businesses of different sizes on a pay per use basis. So successful has the SaaS platform been that it is now used by nearly every software development company as well as most of the world’s biggest companies as well.

To give you some idea of the enormous growth of the SaaS model, here are a few facts and figures:

“SaaS sales in 2010 reached $10 billion and were projected to increase to $12.1 billion in 2011, up 20.7% from 2010.[9] Gartner Group estimates that SaaS revenue will be more than double its 2010 numbers by 2015 and reach a projected $21.3 billion. Customer relationship management (CRM) continues to be the largest market for SaaS. SaaS revenue within the CRM market was forecast to reach $3.8 billion in 2011, up from $3.2 billion in 2010” – Wikipedia

Few other industries could dream of these kinds of growth figures. Even within the IT sector they are impressive and have been instrumental in the record growth of leading companies such as Amazon. They have also helped IT companies such as Microsoft retain their market position, even at a time when sales of its flagship operating Windows system are floundering and it has had to swallow enormous losses from its failed smartphone business.

What characterizes the SaaS model?

  • Gives users access to a vast range of software and services
  • Allows multiple user access to same applications/data at the same time
  • Service is on a subscription basis where users can maximize the return for their money
  • Multiple server framework offers huge scalability
  • Automatic updates and extensive security protection
  • Reduced the strain on user’s hardware resources
  • Reduces the need for expensive licenses
  • Ability to remote access

Though many of these features are common to the IaaS and PaaS models, it is the ability to access software on demand that is the core of the SaaS model.

The main challenges of SaaS integration

2012_bsl_challengecloudsaas_09

The enormous explosion in the popularity of the SaaS platform has in many ways also helped tarnish its reputation. Early SaaS integration with the unsophisticated API’s of the time was often nothing less than a nightmare. I am not going to argue about the many difficulties that implementation of this model has caused people, but will only say that I think anyone who expected such a new technology to emerge without it suffering from major difficulties is more than a little naive.

I would make a point of arguing that the biggest problem the IT sector has ever faced and continues to face is the issue of compatibility and as a complex evolving system the cloud is no different. In regards to the SaaS model; to illustrate the problem of trying to understand so many different types of data sources, I use the simple example of taking a 7-year-old from every country in the world and asking them to translate a few simple sentences from their own language into English. Just imagine how different and indeed confusing their answers would be and you will begin to understand the scope of the problem.

Managing a SaaS cloud platform when it is up and running is as easy as any other kind of platform. The real challenge comes during the initial integration and the problems that surface during the teething stage of using the new architecture.

What are the main issues companies will face during SaaS integration?

  • Data compatibility issues
  • Data corruption during initial transfer
  • Majority of SaaS applications are not designed to synchronize data from multiple lines of business
  • Problems relating to identity and access management
  • API management tools are often insufficient
  • Security breaches relating to data transfer
  • Resource issues such as network overstress

The good news is that as SaaS evolves the solutions we use to overcome many of these problems are evolving too. In fact, according to Blue Mountain Labs founder and CTO David Linthicum, “The technology is there. We just don’t have the strategic knowledge [or] know how all these pieces and parts work and play with our existing infrastructure, processes, data — all the things we have to keep track of,”. Source – Search Cloud Applications blog.

Cloud integration: Strategy for implementation

In his article ‘5 steps to guide your cloud integration strategy’, David Hoff outlines 5 key approaches to successful cloud integration:

  1. Adopt a business-centric approach
  2. Emphasize the importance of data quality
  3. Think enterprise-wide
  4. Consider external requirements
  5. Establish a well-defined governance structure

I would argue that there should be one major addition to his list which I will talk about here:

flexibility-1

  1. Flexibility

The ability to adapt plans and implementation models quickly and effectively is key to any successful implementation. In my time, I have seen too many examples of unsuccessful or really painful cloud integration attempts that resulted from teams doggedly sticking to set migration plans, even in the face of overwhelming problems. I recognize that this often results from a failure in the chain of command, where despite having clearly identified the problem, lower level technicians feel unable to overrule those in control of the project.

I apply the term flexibility to any aspect of the project’s management/implementation that can and should adapt in order to ensure the overall success of the project. This can be anything from replacing ineffective API’s to adjusting timelines and team member’s roles/tasks (though the latter should be a last resort).

To David Hoff’s point regarding the need to ‘establish a well-defined governance structure’, I would add the need to ensure staff are fully trained, competent, and knowledgeable in what they are required to do. An obvious point I know, but one that is often overlooked in an attempt to save money.

The two main types of SaaS integration:

  1. Cloud application integration

When you take a moment to consider just how many different requirements companies and individuals demand of applications, it is not hard to see why application developers are in a constant battle just to keep up.

The main challenge of application integration is that in today’s world, most applications interact in some form or another with other applications. Examples of this can be anything from stock changes being transferred between itinerary programs as orders are fulfilled to customer service agent’s notes being transferred to applications used by sales reps.

I remember a rather nasty incident when during my first job after leaving university, I took a call and finalized an advertisement someone wished to place in our company’s publication. Well, no less than 30 minutes later I was summoned to a meeting with the head of both my department and that of customer services. I was accused of deliberately stealing a co-workers client (and therefore her commission) and was threatened with disciplinary action. I protested my innocence but to no avail. It wasn’t until both departments examined the customer’s file on their respective systems and discovered that due to some application update issue, the notes had not been transferred to my application that my innocence was proved. I give this example to highlight just one example of the consequences that this kind of data transfer problem can cause within a company.

Well, unfortunately, many years later and latency is still a huge issue. Cloud developers have, however, come up with a range of tools to try to overcome this and other issues. These include asynchronous ‘callbacks’ where data is transferred to a pre-established location that is being monitored by the recipient application, and the more scalable ‘send and forget strategy’, which is more straightforward to implement and so is a better alternative for many kinds of companies.

The task of cloud application implementation can be made a lot easier if you are intending to simply transfer over from a ‘traditional application’ over to its cloud counterpart (Microsoft Word for example). Since the data source will be ‘well-known’ to the cloud application this kind of migration is usually quite straightforward.

If you intend to make the transfer from a non-traditional application to a non-related cloud application then you will need to do your homework. You should research compatibility as well as general integration issues such as the data transfer method. If in doubt, give your cloud platform’s customer service agents a call so you can find out for sure.

  1. Cloud data integration

Data integration usually refers to the transfer and synchronization of large amounts of data between repositories where it is stored for later analysis. Since this data is being processed by very few applications and is not subject to real-time alterations, latency and other such problems are not really an issue.

blog-banner-download-project-specs-2

The real hurdles come with the mass transfer of huge amounts of data which can often clog networks and eat up all the available bandwidth. Feasibility studies are essential before attempting to implement this kind of integration. These studies should consider the aspects of existing network architecture that are required to transfer such large amounts of data to evaluate if they are suitable or will need upgrading.  There are many solutions to these issues including implementing ‘Cross Connect Architecture’ and ‘Data integration Solutions’ to help ensure efficient bandwidth allocation and to help clean/standardize data before it is transferred.


Using Cloud Integration Patterns to overcome integration problems

Cloud integration patterns are commonly used to negate integration problems. They allow the creation of a framework to design and implement integrations between different applications, environments, and networks. Since these patterns, once created, are available to other users, they dramatically reduce problems and so offer great cost and time reductions.

Main types of cloud integration patterns:

  • Shared Databases – Designed to facilitate data integration between multiple applications
  • Asynchronous Messaging – Source application is not connected to the receiver(s) application. Rather data is sent to a predetermined destination that is monitored by the receiver application(s). Once detected it will then be transferred to the receiver’s database.
  • Remote Procedure Call – A protocol facilitates data requests by programs on the network
  • File Transfers – Allows the mass transfer of data that is then available to multiple users and multiple applications.


My Final Thoughts

While it is true that cloud integration is certainly no easy task, the main ingredient to success is to keep focused on the overall long-term benefits that it will bring to your business.

If you are at a stage (which most medium to large sized companies should be), where you are strongly considering a move to partial or full cloud integration, then in my view, the sooner the better. The best advice anyone can offer in regards to cloud integration is to keep in mind your goals, to do lots of research, make sure to get lots of second opinions, and always remember to build flexibility into your cloud integration strategy. If you do all that, you are certain to succeed.

Just in case you should run into difficulties and feel like you want to destroy every computer in sight, here is a reminder that you are not alone.

Suggested further reading:

I recommend the following articles as further reading to plug the gaps that time constraints have forced on me. They will help broaden the scope of my article and hopefully help you define an even better cloud integration strategy.

A strategic approach to cloud integration by Rob Fox

Five Ways to Simplify Cloud Integration

The 4 Cloud Integration Dimensions: The Architects Guide to Avoiding Silos by Richard Seroter

 

Aran Davies

Aran Davies

Freelance Writer Developer Photographer
Aran Davies

Latest posts by Aran Davies (see all)