If you want to minimize the amount of money you spend on Amazon Web Services (AWS) infrastructure, then this blog post is for you. In this post I will be discussing the rationale behind starting & stopping AWS instances in an automated fashion and more importantly, doing it in a correct way. Obviously you could do it through the web console of AWS as well, but it will need your daily involvement. In addition, you would have to take care of starting/stopping various services running on those instances.
Before directly jumping on how we achieved instance management in an automated fashion, I like to state the problem that we were facing. Our application testing infrastructure is on AWS and it is a multiple components(20+) application distributed among 8-9 Amazon instances. Usually our testing team starts working from 10 am, and continues till 7 pm. Earlier we used to keep our testing infrastructure up for 24 hours, even though we were using it for only 9 hours on weekdays, and not using it at all on weekends. Thus, we were wasting more then 50% of the money that we spent on the AWS infrastructure. The obvious solution to this problem was: we needed an intelligent system that would make sure that our amazon infrastructure was up only during the time when we needed it.
The detailed list of the requirements, and the corresponding things that we did were:
Once we had the requirements ready, implementing them was simple, as Amazon provides a number of APIs to achieve this. We used AWS CLI, and needed to use just 2 simple commands that AWS CLI provides.
The command to start an instance :
aws ec2 start-instances --instance-ids i-XXXXXXXX
The command to stop an instance :
aws ec2 stop-instances --instance-ids i-XXXXXXXX
Through above commands you can automate starting and stopping AWS instances, but you might not be doing it the correct way. As you didn't restrict the AWS CLI allow firing of start-instances and stop-instances commands only, you could use other commands and that could turn out to be a problem area. Another important point to consider is that we should restrict the AWS instances on which above commands could be executed, as these commands could be mistakenly run with the instance id of a production amazon instance id as an argument, creating undesirable circumstances :)
In the next blog post I will talk about how to start and stop AWS instances in a correct way.
Before directly jumping on how we achieved instance management in an automated fashion, I like to state the problem that we were facing. Our application testing infrastructure is on AWS and it is a multiple components(20+) application distributed among 8-9 Amazon instances. Usually our testing team starts working from 10 am, and continues till 7 pm. Earlier we used to keep our testing infrastructure up for 24 hours, even though we were using it for only 9 hours on weekdays, and not using it at all on weekends. Thus, we were wasting more then 50% of the money that we spent on the AWS infrastructure. The obvious solution to this problem was: we needed an intelligent system that would make sure that our amazon infrastructure was up only during the time when we needed it.
The detailed list of the requirements, and the corresponding things that we did were:
- We should shut down our infrastructure instances when we are not using them.
- There should be a functionality to bring up the infrastructure manually: We created a group of Jenkins jobs, which were scheduled to run at a specific time to start our infrastructure. Also a set of people have execution access to these jobs to start the infrastructure manually, if the need arises.
- We should bring up our infrastructure instances when we need it.
- There should be a functionality to shut down the infrastructure manually: We created a group of Jenkins jobs that were scheduled to run at a specific time to shut down our infrastructure. Also a set of people have execution access on these jobs to shut down the infrastructure manually, if the need arises.
- Automated application/services start on instance start: We made sure that all the applications and services were up and running when the instance was started.
- Automated graceful application/services shut down before instance shut down: We made sure that all the applications and services were gracefully stopped before the instance was shut down, so that there was be no loss of data.
- We also had to make sure that all the applications and services should be started as per defined agreed order.
Once we had the requirements ready, implementing them was simple, as Amazon provides a number of APIs to achieve this. We used AWS CLI, and needed to use just 2 simple commands that AWS CLI provides.
The command to start an instance :
aws ec2 start-instances --instance-ids i-XXXXXXXX
The command to stop an instance :
aws ec2 stop-instances --instance-ids i-XXXXXXXX
Through above commands you can automate starting and stopping AWS instances, but you might not be doing it the correct way. As you didn't restrict the AWS CLI allow firing of start-instances and stop-instances commands only, you could use other commands and that could turn out to be a problem area. Another important point to consider is that we should restrict the AWS instances on which above commands could be executed, as these commands could be mistakenly run with the instance id of a production amazon instance id as an argument, creating undesirable circumstances :)
In the next blog post I will talk about how to start and stop AWS instances in a correct way.
Thanks for this excellent article :) Keep posting.
ReplyDelete- Nandini,
Trainer,
Kamal Technologies - Best Node JS training institute in Chennai
Needed to compose you a very little word to thank you yet again regarding the nice suggestions you’ve contributed here. aws training in chennai
ReplyDelete
ReplyDeleteI've been surfing on the web more than 3 hours today, yet I never found any stupefying article like yours. It's imperatively worth for me. As I would see it, if all web proprietors and bloggers made confusing substance as you did, the net will be in a general sense more profitable than at whatever point in late memory.
Digital Marketing Training in Mumbai
Six Sigma Training in Dubai
Six Sigma Abu Dhabi
I think this is a great site to post and I have read most of contents and I found it useful for my Career .Thanks for the useful information. For any information or Queries Comment like and share it.
ReplyDeletePMP Training Abu Dhabi
GDPR Training in Hyderabad
Digital Marketing Training in Hyderabad
six sigma Training Pune
Very nice post here and thanks for it .I always like and such a super contents of these post.Excellent and very cool idea and great content of different kinds of the valuable information's.
ReplyDeleteGood discussion. Thank you.
Anexas
Six Sigma Training in Abu Dhabi
Six Sigma Training in Dammam
Six Sigma Training in Riyadh
Amazon has a simple web services interface that you can use to store and retrieve any amount of data, at any time, from anywhere on the web. Amazon Web Services (AWS) is a secure cloud services platform, offering compute power, database storage, content delivery and other functionality to help businesses scale and grow.For more information visit.
ReplyDeleteaws online training
aws training in hyderabad
amazon web services(AWS) online training
amazon web services(AWS) training online