COST-MINIMIZING DYNAMIC MIGRATION OF CONTENT DISTRIBUTION SERVICES INTO HYBRID CLOUDS

COST-MINIMIZING DYNAMIC MIGRATION OF CONTENT

DISTRIBUTION SERVICES INTO HYBRID CLOUDS

Abstract—With the recent advent of cloud computing technologies, a growing number of content distribution applications are contemplating a switch to cloud-based services, for better scalability and lower cost. Two key tasks are involved for such a move: to migrate the contents to cloud storage, and to distribute the web service load to cloud-based web services. The main issue is to best utilize the cloud as well as the application provider’s existing private cloud, to serve volatile requests with service response time guarantee at all times, while incurring the minimum operational cost. While it may not be too difficult to design a simple heuristic, proposing one with guaranteed cost optimality over a long run of the system constitutes an intimidating challenge. Employing Lyapunov

optimization techniques, we design a dynamic control algorithm to optimally place contents and dispatch requests in a hybrid cloud infrastructure spanning geo-distributed data centers, which minimizes overall operational cost over time, subject to service response time constraints. Rigorous analysis shows that the algorithm nicely bounds the response times within the preset QoS target, and guarantees that the overall cost is within a small constant gap from the optimum achieved by a T-slot look ahead mechanism with known future information. We verify the performance of our dynamic algorithm with prototype-based evaluation.

EXISTING SYSTEM:

Migration of applications into clouds: A number of research projects have emerged in recent years that explore the migration of services into a cloud platform. develop an optimization model for migrating enterprise IT applications onto a hybrid cloud. Their model takes into account enterprise-specific constraints, such as transaction delays and security policies. Onetime optimal service deployment is considered, while our work investigates optimal dynamic migration over time, to achieve the long-term optimality.   In epropose an intelligent algorithm to factor workload and dynamically determine the service placement across the public cloud and the private cloud. Their focus is on designing an algorithm for distinguishing base workload and trespassing workload. Migration of content delivery services into clouds: Some research efforts have been put into migrating generic content delivery services onto clouds. MetaCDN by Pathan et al.  a proof-of-concept testbed, experiments on which show that deploying content delivery based on storage clouds can improve utility, based on primitive content placement and request routing mechanisms. Chen  propose to build CDNs in the cloud in order to minimize cost under the constraints of QoS requirement, but they only propose greedy-strategy based heuristics without provable properties. In contrast, we target an optimization framework which renders optimal migration solutions for long run of the system.

PROPOSED SYSTEM:

The contribution of this work can be summarized as follows:

·        We propose a generic optimization framework for dynamic, optimal migration of a content distribution service to a hybrid cloud consisting of a private cloud and public geo-distributed cloud services.

·        We design a joint content placement and load distribution algorithm for dynamic content distribution service deployment in the hybrid cloud. Providers of content distribution services can practically apply it to guide their service migration, with confidence in cost minimization and performance guarantee, regardless of the request arrival pattern.

·        We demonstrate optimality of our algorithm with rigorous theoretical analysis and prototype-based evaluation. The algorithm nicely bounds the response times (including queueing and round-trip delays) within the preset QoS target in cases of arbitrary request arrivals, and guarantees that the overall cost is within a small constant gap from the optimum achieved by a T-slot lookahead mechanism with information into the future.

Module 1

Hybrid Cloud

A hybrid cloud is a combination of a private cloud combined with the use of public cloud services where one or several touch points exist between the environments. The goal is to combine services and data from a variety of cloud models to create a unified, automated, and well-managed computing environment. Combining public services with private clouds and the data center as a hybrid is the new definition of corporate computing. Not all companies that use some public and some private cloud services have a hybrid cloud. Rather, a hybrid cloud is an environment where the private and public services are used together to create value.

A cloud is hybrid

·         If a company uses a public development platform that sends data to a private cloud or a data center–based application.

·         When a company leverages a number of SaaS (Software as a Service) applications and moves data between private or data center resources.

·         When a business process is designed as a service so that it can connect with environments as though they were a single environment.

Module 2

Dynamic Migration

Currently, many Web services have been deployed by  different organizations that are widely distributed over the  Internet. These are mostly software services running on fixed hardware resources. When composing multiple services for a system, it is likely that some selected software services are hosted at widely distributed sites. This brings potential performance problems. Sending a service request along with a large quantity of input data across the

wide area network can be costly. It increases the network traffic and raises the potential of unexpected delays due to network congestions. This can be a major barrier for applications that have real-time requirements. For example, a commander may dynamically assemble a command and control application that involves a large number of web  services, such as many data services based on continuous input from the remote sensors, image processing services, information fusion services, etc. to assist her/his decision making. Communication among two data processing services may involve a large amount of data and may result in delays due to network congestions. Such delays can affect the timeliness of the decision and cause costly consequences.   However, if there are a limited number of services to choose from, it may be difficult to significantly reduce the communication latency. In cloud environment, this problem can be solved by considering service migration. One of major advances in cloud environment is that computing hardware resources and their management utilities are all provided as services. The widely distributed computing resources can be used to host migrated services to potentially minimize the communication cost. However, not all services can be migrated. Services based on hardware resources are less flexible and cannot be  igrated (not in the cyber world). When the services involve common hardware devices, the devices, even though non-migratable, are likely to be all over the place. Thus, it is possible to select one that can result in minimized communication cost. When a service involves specialized hardware, then it cannot be migrated. Services can potentially be migrated, but the migration costs and gains have to be evaluated to ensure net performance gains.   

Module 3

The service migration problem

System Model We consider a typical content distribution application, which provides a collection of contents (files), denoted as set M, to users spreading over multiple geographical regions. There is a private cloud owned by the provider of the content distribution application, which stores the original copies of all the contents. The private cloud has an overall upload bandwidth of b units for serving contents to users. There is a public cloud consisting of data centers located in multiple geographical regions, denoted as set N. One data center resides in each region. There are two types of inter-connected servers in each data center: storage servers for data storage, and computing servers that support the running and provisioning of virtual machines (VMs). Servers inside the same data center can access each other via a certain DCN (Data Center Network). The provider of the content distribution application (application provider) wishes to provision its service by exploiting a hybrid cloud architecture, which includes the geo-distributed public cloud and its private cloud. The major components of the content distribution application include: (i) back-end storage of the contents and (ii) front-end web service that serves users’ requests for contents. The application provider may migrate both service components into the public cloud: contents an be replicated in storage servers in the cloud, while requests can be dispatched to web services installed on VMs on the computing servers.

Module 4

Cost-Minimizing Service Migration Problem

We suppose that the system runs in a time-slotted fashion. Each time slot is a unit time which is enough for uploading any file m 2 M with size v(m) (bytes) at the unit bandwidth. In time slot t, a(m) j (t) requests are generated for downloading file m 2 M, from users in region j. We assume that the request arrival is an arbitrary process over time, and the number of requests arising from one region for a file in each time slot is upper-bounded by Amax. The cost of uploading a byte from the private cloud is h. The charge for storage at data center i is pi per byte per unit time. gi and oi per byte are charged for uploading from and downloading into data center i, respectively. The cost for renting a VM instance in data center i is fi per unit time. These charges follow the charging model of leading commercial cloud providers, such as Amazon EC2  and S3. We assume that the storage capacity in each data center is sufficient for storing contents from this content distribution application. We also assume that each request is served at one unit bandwidth, and the number of requests that a VM in data center i can serve per unit time.

Module 5

Dynamic migration algorithm

In this section, we design a dynamic control algorithm using Lyapunov optimization techniques, which solves the optimal migration problem in  and bounds the time-averaged round-trip delays and queueing delays for each request. We also discuss its practical implementation. Bounding Delays The optimization problem  includes a constraint on time-averaged variable values, i.e., inequality. Our dynamic algorithm will only be able to adjust variables in each time slot. How can we guarantee this inequality by controlling the variable values over time?

To satisfy constraint , we resort to the virtual queue techniques in Lyapunov optimization.

CONCLUSION

This paper investigates optimal migration of a content distribution service to a hybrid cloud consisting of a private cloud and public geo-distributed cloud services. We propose a generic optimization framework based on Lyapunov optimization theory, and design a dynamic, joint content placement and request distribution algorithm, which minimizes the operational cost of the application with QoS guarantees. We theoretically show that our algorithm approaches the optimality achieved by a mechanism with known information in the future T time slots by a small gap, no matter what the request arrival pattern is. Our prototype-based evaluation verifies our theoretical findings. We intend to extend the framework to specific content distribution services with detailed requirements, such as video-on-demand services or social media applications, in our ongoing work.

REFERENCES

[1] Amazon CloudFront, http://aws.amazon.com/cloudfront/.

[2] Microsoft Azure, http://www.microsoft.com/windowsazure/.

[3] Google App Engine, http://code.google.com/appengine/.

[4] Dropbox, http://www.dropbox.com/.

[5] Microsoft Office Web Apps, http://office.microsoft.com/enus/ web-apps/.

[6] Google docs, http://docs.google.com/.

[7] M. Hajjat, X. Sun, Y. E. Sung, D. Maltz, and S. Rao, “Cloudward Bound: Planning for Beneficial Migration of Enterprise Applications to the Cloud,” in Proc. of IEEE SIGCOMM, August 2010.

[8] H. Zhang, G. Jiang, K. Yoshihira, H. Chen, and A. Saxena, “Intelligent Workload Factoring for a Hybrid Cloud Computing Model,” in Proc. of the International Workshop on Cloud Services (IWCS 2009), June 2009.

[9] H. Li, L. Zhong, J. Liu, B. Li, and K. Xu, “Cost-effective Partial Migration of VoD Services to Content Clouds,” in Proc. of IEEE CLOUD, July 2011.

[10] X. Cheng and J. Liu, “Load-Balanced Migration of Social Media to Content Clouds,” in Proc. of NOSSDAV, June 2011.

[11] L. Georgiadis, M. J. Neely, and L. Tassiulas, “Resource allocation and cross-layer control in wireless networks,” Foundations and Trends in Networking, vol. 1, no. 1, pp. 1–149, 2006.

[12] M. J. Neely, Stochastic Network Optimization with Application to Communication and Queueing Systems. Morgan & Claypool, 2010.

[13] “Energy optimal control for time varying wireless networks,” IEEE Tran. on Information Theory, no. 7, pp. 2915–2934, July 2006.

[14] M. M. Amble, P. Parag, S. Shakkottai, and L. Ying, “Content- Aware Caching and Traffic Management in Content Distribution Networks,” in Proc. of IEEE INFOCOM, April 2011.